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EXECUTIVE  SUMMARY 


RT-19A,  Research  on  Building  Education  &  Workforce  Capacity  in  Systems  Engineering,  is  the 
second  phase  of  a  two-year  research  study  whose  goal  is  to  understand  the  impact  of  diverse 
capstone  courses  that  exposed  undergraduate  and  graduate  engineering  majors  to  authentic 
Department  of  Defense  (DoD)  problems  and  engaged  them  in  the  learning  and  practice  of 
systems  engineering,  and  outcomes  related  to  systems  engineering  careers  and  interest.  Over 
an  18-month,  three-phase  effort  from  April  2011  to  September  2012  that  encompassed  course 
planning,  implementation,  and  analysis,  participating  RT-19A  schools  and  the  research  team 
explored  methods  and  approaches  to  augment  the  systems  engineering  workforce  for  future 
DoD  and  related  industry  workforce  needs. 

The  strategic  goals  addressed  by  this  research  are  twofold:  to  understand  the  institutional 
challenges  and  successes  in  the  adoption  of  core  elements  of  successful  systems  engineering 
capstone  projects;  and  to  examine  the  contexts  and  program  characteristics  leading  to  highly 
successful  student  team-developed  products  and  artifacts  that  respond  to  authentic 
Department  of  Defense  (DoD)  problem  areas.  To  produce  the  following  report,  the  research 
team  gathered  data  from  student  pre  and  post  surveys  in  order  to  analyze  the  impact  of  the 
systems  engineering  capstone  project  on  student  learning  of  systems  engineering,  student 
interest  in  systems  engineering  careers,  and  student  awareness/interest  in  authentic  DoD 
problems.  In  addition,  this  report  also  contains  input  gathered  from  surveys  submitted  by  Pis 
and  mentors,  and  from  observations  and  interviews  taken  from  a  systems  engineering  capstone 
conference  June  2012. 

Institutions  were  selected  for  participation  through  a  competitive  application  process  based  on 
a  set  of  criteria  developed  in  consultation  with  the  sponsor,  and  partners  were  awarded  a 
subcontract  for  the  development,  implementation,  analysis,  and  reporting  on  their  systems 
engineering  capstone  project.  Altogether,  sixteen  schools  were  selected  to  participate  in  the  RT- 
19A  effort:  six  Systems  Engineering  Research  Center  (SERC)  member  universities,  four  service 
academies,  and  six  partner  schools.  In  the  first  year  of  this  study  fifteen  systems  engineering 
capstone  courses  were  developed  and  implemented  at  six  military  institutions  and  eight  civilian 
universities  affiliated  with  the  Systems  Engineering  Research  Center.  Ten  of  those  schools 
returned  to  participate  in  this  year's  effort. 

The  capstone  courses  were  organized  around  SPRDE-SE  systems  engineering  competencies  and 
selection  of  Department  of  Defense  problem  areas.  Five  topic  areas  illustrating  authentic  DoD 
problems  were  presented  for  student  teams'  projects.  Problem  areas  #2  and  4  (see  Table  3  for 
more  complete  description)  were  the  most  researched  topics,  with  more  than  half  the  projects 
addressing  one  of  the  two  problem  areas.  Selection  of  problem  areas  was  based  on  student 
research  interest,  expertise  of  participating  faculty,  or  the  decision  to  continue  capstone  designs 
from  the  prior  year.  Institutions  organized  their  teams  in  different  ways.  The  most  common 
structure  included  several  teams  each  working  on  a  subsystem. 

According  to  final  reports  submitted  by  principal  investigators,  306  and  339  students 
participated  in  RT-19A-sponsored  systems  engineering  capstone  courses  in  the  fall  2011  and 
spring  2012  semesters,  respectively.  Many  institutions  enrolled  the  same  students  for  both 
semesters.  An  estimated  198  students  worked  on  DoD  problem  areas,  or  64.7%  of  students 
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enrolled  in  the  spring  courses.  Of  these,  154  were  undergraduates,  38  were  graduate  students, 
and  1  was  a  postgraduate  student.  The  population  was  over  three-quarters  male  and  20% 
female.  Thirty  percent  of  the  students  surveyed  were  systems  engineering  majors,  followed  by 
mechanical  engineering  majors  (25%).  Other  student  majors  included  electrical,  industrial, 
software,  and  civil  engineering;  computer  science;  and  engineering  management.  Only  half  of 
the  students  reported  working  in  multidisciplinary  teams  prior  to  their  capstone  experience. 

Fifty-one  faculty  members  participated  in  the  development,  delivery,  and  assessment  of  RT-19A 
courses,  almost  the  same  number  as  participated  in  RT-19  (50),  with  the  highest  percentage 
from  Mechanical  Engineering  departments,  followed  by  Systems  Engineering.  This  year's  faculty 
also  came  from  Industrial,  Electrical,  Civil,  Mechanical,  Systems,  Software,  Ocean  Engineering, 
and  Computer  Science.  Eight  schools  included  faculty  participants  from  more  than  one 
engineering  discipline. 

Over  the  course  of  two  semesters,  students  enrolled  in  the  capstone  courses  created  a  number 
of  physical  prototypes,  summarized  in  Table  5,  that  responded  to  their  DoD  problem  areas. 
Overall,  75%  of  responding  students  felt  their  team  produced  projects  that  successfully  fulfilled 
requirements;  showed  proof-of-concept;  encouraged  multidisciplinary,  intergroup 
communication  and  coordination;  and  demonstrated  their  understanding  of  systems 
engineering  concepts,  from  the  initial  design  and  requirements  determination  stages,  to  final 
prototype  testing.  Of  those  who  did  not  feel  their  projects  were  successful,  lack  of  resources  and 
time  were  the  most  frequently  cited  reasons.  The  students  attributed  parts  delays;  the  inability 
to  build  an  operational  prototype  or  to  complete  specific  phases  of  the  project,  such  as  testing; 
communication  between  team  members  from  different  disciplines;  and  communication  over 
distance  as  project  problems.  Pis  cited  technical  issues  with  modifying  off-the-shelf  (COTS) 
software  and  hardware  (e.g.,  Microsoft  Kinect,  batteries);  time  management;  delays  in  parts 
acquisition;  budget  limits;  and  funding  delays  as  challenges  to  student  prototype  construction. 

A  goal  of  the  systems  engineering  capstone  courses  implemented  in  RT-19A  was  to  increase 
student  awareness  of  the  diversity  of  problem  areas  addressed  by  the  DoD.  From  pre-  to  post¬ 
survey,  there  was  an  increase  from  14%  to  18%  of  students  who  listed  what  were  clearly 
systems  engineering  issues  ("requirements  management,"  "project  scheduling,"  "systems 
integration,"  "predictive  decision  algorithms").  Research  related  to  military  field  needs 
(materials  research,  troop  protection,  expeditionary  housing,  water  filtration,  improved  IED 
detection,  and  lightweight  armor)  increased  the  most  in  students'  awareness. 

Another  goal  of  the  systems  engineering  capstone  courses  was  to  increase  student  interest  in 
systems  engineering  careers  generally;  systems  engineering  careers  in  government;  and  systems 
engineering  careers  in  industry.  A  comparison  was  made  between  the  means  of  the  baseline 
survey  respondents  and  post-survey  matched  group  in  all  three  categories.  Results  indicated 
that  the  matched  group  was  biased  toward  systems  engineering  careers  from  the  beginning, 
with  higher  mean  scores  on  the  baseline  survey  than  the  larger  group  of  respondents.  Post¬ 
survey  means  for  the  entire  population  of  matched  pre/post-survey  responses  decreased  in  all 
three  categories,  although  these  decreases  were  not  statistically  significant  and  none  of  the 
means  were  less  than  "3,"  indicating  a  moderate  interest  in  systems  engineering  careers. 

Further  analyses  of  students'  responses  show  more  subtle  differences  in  the  level  of  interest 
(from  low  to  high)  among  the  various  subgroups  analyzed.  Where  there  was  change  was  in  the 
mean  scores  for  those  who  chose  1,  2,  or  3  on  the  5-point  scale  in  the  baseline  survey.  This 
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change  was  statistically  significant  in  the  direction  of  increased  student  interest  in  becoming  a 
systems  engineer  for  government. 

An  important  component  of  the  capstone  experience  was  the  inclusion  of  mentors  who  played 
multiple  roles  as  technical  experts  guiding  student  teams  toward  solutions  and  risk  assessment; 
as  reviewers  at  interim  and  final  design  presentations;  as  clients  who  helped  determine 
requirements;  and  as  career  advisors.  Pis  reported  participation  of  over  forty  mentors  from 
Department  of  Defense  offices  and  other  leading  defense  industry  corporations  (a  full  list  is 
included  in  Appendix  B  of  the  report).  All  institutions  that  implemented  capstone  courses  had 
DoD-assigned  mentors  in  place  before  the  start  of  the  school  semester,  with  the  exception  of 
two  partner  schools  that  did  not  have  assignments,  and  one  lead  institution  that  utilized  an 
advisory  board  of  industry  professionals.  Similar  to  DoD-assigned  mentors,  industry  mentors 
worked  with  students  in  all  schools,  excluding  the  two  aforementioned  partner  schools,  on 
specific  problem  areas  (e.g.,  assistive  or  immersive  training  technologies,  systems  assurance, 
among  others).  Both  DoD  and  industry  mentors  visited  students  on  campus  periodically, 
attended  design  reviews,  and  communicated  with  teams  through  email,  phone,  and  video  and 
teleconference. 

Three-quarters  of  mentor  survey  respondents  gave  student  projects  high  rankings  for  meeting 
their  goals.  Mentors  reported  wanting  both  formal  and  informal  opportunities  to  communicate 
with  students;  however,  scheduling  conflicts  were  cited  as  the  primary  barrier  to  increased 
engagement.  Almost  90%  of  surveyed  students  felt  that  mentor  feedback  had  helped  them  with 
their  projects.  Students  recommended  that  mentors  interact  with  teams  earlier  in  the  semester; 
guide  teams  towards  inquiry-based  solutions;  and  set  realistic  expectations  for  projects. 
Beneficial  impacts  of  mentor  involvement  were  reported  by  Pis  when  communication  was 
frequent,  specific,  and  initiated  early  in  the  semester.  Three-quarters  of  Pis  interviewed  in  the 
final  survey  described  mentorships  as  highly  successful  and  efficient;  in  one  instance,  the 
intervention  of  a  mentor  was  critical  to  a  partnership,  providing  much  needed  clarification  and 
encouragement  for  a  student  team  that  struggled  to  understand  its  role  in  providing  systems 
assurance  for  another  school  located  several  time  zones  away. 

Through  site  visits  to  systems  engineering  capstone  universities  in  spring  2011,  a  team  of 
sponsor  representatives  had  identified  nine  promising  practices— approaches  that  were  present 
at  universities  where  students  demonstrated  higher  levels  of  communication,  analysis,  and 
awareness  of  the  systems  engineering  process  during  the  site  visits.  This  year,  all  institutions 
incorporated  three  or  more  of  the  practices  into  their  capstone  courses.  A  graphical 
representation  of  the  presence  (or  lack  thereof)  of  these  promising  practices  among  all 
participating  RT-19A  universities  appears  in  Table  32.  The  formation  of  cross-disciplinary  teams, 
regular  involvement  with  mentors,  and  attendance  by  mentors  at  student  design  reviews  were 
practices  adopted  by  nearly  all  of  the  participating  schools.  The  recommendation  to  organize 
the  fall  lecture-based  course,  and  to  commence  prototype  design  in  the  spring,  was  not 
implemented.  Pis  reported  that  they  worked  on  DoD  problem  areas  and  simultaneously 
delivered  engineering  instruction  in  order  to  accommodate  the  academic  calendar  and  also  to 
coordinate  research,  materials,  and  personnel. 

Another  defining  characteristic  of  the  RT-19A  capstone  experience  was  scaling  up  to  include  five 
partnerships  between  a  total  of  eleven  schools.  The  report  describes  in  detail  capstone 
partnerships  conducted  over  distance  between  service  academies,  civilian  schools,  and  schools 
with  and  without  systems  engineering  programs.  The  five  partnership  models  were  each 
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qualitatively  different,  with  models  organized  around  the  co-development  of  prototypes 
between  teams  or  the  delegation  of  one  part  of  the  systems  engineering  process  to  a  remotely 
based  team.  Another  type  of  partnership  dealt  exclusively  with  faculty  professional 
development  at  two  schools  and  had  no  direct  student  team  collaboration.  The  coordination  of 
communication  among  students  in  different  engineering  disciplines  and  geographical  locations 
was  one  issue  that  impacted  many  of  the  partnerships  that  were  geographically  and  temporally 
distant.  Timely  funding  to  support  resource  and  capacity;  existing  faculty  relationships  between 
partnering  schools;  understanding  of  differences  in  school  culture;  consistent  communication 
between  partnering  faculty  members;  complementary  knowledge/skills  and  complementary 
research  interests  to  ensure  that  all  areas  of  expertise  are  covered  within  the  collaboration;  and 
student  interest  in  DoD  problem  areas  all  contributed  to  successful  partnerships.  Pis  reported 
that  partner  schools  benefited  from  initial  "meet  and  greet"  sessions  with  students,  mentors, 
and  faculty  to  promote  the  collaboration;  however,  such  relationships  had  to  be  continually 
encouraged  and  maintained. 

Findings  and  Recommendations: 

Limitations  in  the  data  and  the  many  approaches  and  variables  used  in  the  15  capstone  courses 
prevent  statistical  correlations  with  student  outcomes  and  "optimal"  course  designs.  However, 
the  following  summary  of  findings  are  grounded  in  data  collected  through  RT-19A: 

Students  enjoyed  the  real-world  nature  of  the  projects— both  in  terms  of  building  an  artifact 
that  might  be  used  and  in  terms  of  the  systems  engineering  project  context  (budget  constraints, 
interdisciplinary  teams,  experts  as  mentors)— and  that  they  appreciated  the  contribution  that 
the  systems  engineering  perspective  brought  to  their  work.  Mentorships  and  partnerships  were 
an  integral  part  of  this  year's  effort,  and  required  management  by  faculty  to  coordinate 
communication  and  increase  student  content  knowledge  of  systems  engineering  concepts. 

Systems  engineering  capstone  courses  do  not  appear  to  have  had  a  major  impact  on  the 
students'  immediate  career  plans,  although  it  must  be  noted  that  many  had  their  immediate 
post-college  plans  in  place  and  that  a  large  majority  of  both  undergraduates  and  graduate 
students  believed  that  they  might  choose  careers  in  systems  engineering  sometime  in  the 
future.  Although  significant  planning  toward  logistics  management,  funding,  personnel, 
curriculum  design,  and  other  major  decisions  is  required  prior  to  capstone  course 
implementation,  the  majority  of  stakeholders  in  the  research  study  (students,  faculty/PIs,  and 
mentors)  reported  benefits  that  ranged  from  increasing  students'  exposure  to  the  systems 
engineering  process  through  the  investigation  of  real-life  problem  areas;  interaction  with 
mentors  from  a  variety  of  industries;  and  the  facilitation  of  prototype  design  in  multidisciplinary 
teams  and  remote  collaborations. 

Benefits  of  these  school  partnerships  include: 

•  Schools  that  do  not  have  systems  engineering  gain  access  to  schools  with  systems 
engineering  expertise 

•  Students  at  schools  with  only  one  engineering  major  are  able  to  work  in 
multidisciplinary  teams 

•  Students  have  access  to  a  wider  variety  of  student  skills  and  abilities  when  forming 
teams 

•  Students  are  exposed  to  a  wider  diversity  of  teammates 

•  Students  are  exposed  to  a  wider  variety  of  mentors 
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•  Students  at  civilian  schools  gain  access  to  military  commands  and  to  DoD  problem  areas 

•  Students  learn  the  benefits  and  difficulties  of  working  at  a  distance 

New  challenges  introduced  by  these  partnerships  include: 

•  Students  at  different  schools  may  have  different  academic  calendars,  be  in  different 
time  zones,  and  therefore  have  difficulty  coordinating  schedules,  meetings,  delivery 
timelines,  etc. 

•  Students  may  have  difficulty  communicating  at  a  distance 

•  Students  who  cannot  meet  face-to-face  may  have  difficulty  learning  trust,  determining 
roles,  and  developing  collaborations 

The  report  concludes  with  some  suggestions  for  how  these  partnerships  might  be  facilitated  on 
a  national  scale  in  the  future. 


RT  19a  Final  Technical  Report,  SERC-TR-019-2,  September  30,  2012 


10 


UNCLASSIFIED 


1.0  INTRODUCTION 


1.1  PROJECT  OVERVIEW 

A  45%  growth  is  expected  in  systems  engineering  jobs  in  the  next  decade,  and  there  have  been 
numerous  studies  and  workshops  that  have  highlighted  the  shortfalls  in  both  the  number  and 
capability  of  the  systems  engineering  workforce  (Rosato,  Braverman,  &  Jeffries,  2009).  The  July 
2006  National  Defense  Industrial  Association  (NDIA)  Task  Force  noted  among  the  top  five 
systems  engineering  issues  the  lack  of  adequate,  qualified  systems  engineering  human  capital 
resources  within  government  and  industry  for  allocation  on  major  programs  (National  Defense 
Industrial  Association  SE  Division  Task  Group ,  2006).  In  the  July  2010  NDIA  white  paper  on 
critical  systems  engineering  challenges,  Issue  2  was  identified  as:  The  quantity  and  quality  of 
systems  engineering  expertise  is  insufficient  to  meet  the  demands  of  the  government  and 
defense  industry,  and  further  outlined  certain  recommendations  to  build  systems  engineering 
expertise  and  capacity.  In  particular,  it  recommended  developing  systems  engineering  expertise 
through  "role  definition,  selection,  training,  career  incentives,  and  broadening  'systems  thinking' 
into  other  disciplines,"  and  made  a  number  of  specific  recommendations,  including  adding  an 
introductory  course  in  systems  engineering  in  all  undergraduate  engineering  and  technical 
management  degree  programs;  and  working  with  major  universities  to  recommend  systems 
engineering  curricula  to  improve  consistency  across  programs  in  order  to  achieve 
standardization  of  skill  sets  for  graduates  (National  Defense  Industrial  Association  SE  Division, 
2010).  With  these  industry-wide  workforce  demands  challenging  the  systems  engineering 
community,  Research  on  Building  Education  &  Workforce  Capacity  in  Systems  Engineering 
(referred  to  as  the  Systems  Engineering  Capstone  Project)  was  conceptualized  and  designed  to 
pilot  and  evaluate  approaches  to  ameliorating  these  shortages  in  a  select  number  of  systems 
engineering  institutions.  The  first  year  of  the  project,  referred  to  as  RT-19,  took  place  in  2010- 
2011,  resulting  in  a  report  dated  October  31,  2011.  This  current  report  discusses  the  results  of 
the  second  year,  referred  to  as  RT-19A,  whose  aim  was  to  test  the  replication,  scale-up  and 
institutionalization  of  practices,  instructional  strategies,  and  course  materials/resources  judged 
effective  during  the  first  year.  RT-19A  also  introduced  the  concept  of  partner  schools,  which 
were  non-systems  engineering  schools  that  would  partner  with  RT-19  schools  in  order  to  extend 
the  reach  and  impact  of  the  systems  engineering  effort.  The  results  of  both  RT-19  and  RT-19A 
are  to  inform  the  development  of  a  national  scale-up  effort  that  will  substantially  expand  the 
number  and  capabilities  of  universities  that  can  produce  the  systems  engineering  graduates 
needed  for  the  DoD  and  related  defense  industry  workforce. 


1.2  PARTICIPANTS  AND  RESEARCH  SETTING 

As  was  the  case  for  RT-19,  a  request  for  proposals  was  issued  and  a  competitive  application 
process  was  conducted  in  order  to  select  returning  RT-19  institutions,  both  those  that  were 
Systems  Engineering  Research  Center  (SERC)  members  and  the  service  academies,  with 
proposals  that  included  partnership  opportunities  receiving  priority. 

Altogether,  16  schools  were  selected  to  participate  in  the  RT-19A  effort:  six  SERC  member 
universities,  four  service  academies,  and  six  partner  schools.  In  comparison,  14  schools 
participated  in  RT-19,  with  ten  of  those  schools  returning  for  RT-19A. 
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RT-19A  Lead  School 

Partner  School 

Air  Force  Academy 

Auburn  University 

Tuskegee  University 

Coast  Guard  Academy 

Connecticut  College 

University  of  Rhode  Island 

Missouri  University  of  Science  and  Technology 

Military  Academy 

Naval  Academy 

Smith  College 

Naval  Postgraduate  School 

Southern  Methodist  University 

University  of  Hawaii  at  Manoa 

Stevens  Institute  of  Technology 

University  of  Virginia 

Sweet  Briar  College 

Table  1:  Lead  Institutions  with  Partner  Schools 

Research  was  conducted  in  the  context  of  capstone  systems  engineering  courses  ("capstone 
courses")  developed  at  15  of  the  16  schools.  Tuskegee  University  did  not  develop  a  capstone 
course  on  its  campus;  instead,  two  faculty  members  acted  as  observing  partners  for  the 
capstone  course  offered  through  Auburn. 

In  most  cases,  capstone  courses  were  integrative,  culminating,  project-based  experiences  where 
teams  of  students  worked  together  to  develop  a  product  or  prototype  that  addressed  a  DoD 
need,  such  as  low-cost,  low-power  computing  devices;  pre-positioned  expeditionary  assistance 
kits;  expeditionary  housing  systems;  immersive  training  technologies;  and  assistive  technologies 
for  wounded  warriors.  The  goal  was  to  embed,  infuse,  and  augment  systems  engineering 
knowledge  for  undergraduate  and  graduate  students,  as  defined  by  the  Systems  Planning, 
Research  Development,  and  Engineering  (SPRDE)-SE  and  Program  Systems  Engineer  (PSE) 
competency  model,  known  as  the  SPRDE-SE/PSE  Competency  Model  (Table  2). 

As  was  the  case  with  RT-19,  one  of  the  goals  of  RT-19A  was  to  examine  student  learning 
outcomes  resulting  from  systems  engineering  capstone  experiences.  The  Systems  Planning, 
Research  Development,  and  Engineering  Systems  Engineering  and  Program  Systems  Engineer 
(SPRDE-SE/PSE)  competency  model  served  as  the  standard  for  systems  engineering  knowledge 
and  skill  (see  Table  2).  Analysis  of  survey  results  from  primary  investigators,  students,  and 
mentors;  input  from  site  visits  by  the  DoD  DR&E  sponsor;  interviews  with  lead  schools  and  their 
partners;  and  insights  gleaned  from  panels  and  presentations  at  the  culminating  RT-19A 
workshop  provide  the  data  on  which  this  final  report  and  recommendations  are  based. 
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SPRDE-SE  /PSE  Competencies 

Analytical  (13) 

1.  Technical  Basis  for  Cost 

2.  Modeling  and  Simulation 

3.  Safety  Assurance 

4.  Stakeholder  Requirements  Definition  (Requirements  Development) 

5.  Requirements  Analysis  (Logical  Analysis) 

6.  Architectural  Design  (Design  Solution) 

7.  Implementation 

8.  Integration 

9.  Verification 

10.  Validation 

11.  Transition 

12.  System  Assurance 

13.  Reliability,  Availability,  and  Maintainability 

Technical 

Management 

(12) 

14.  Decision  Analysis 

15.  Technical  Planning 

16.  Technical  Assessment 

17.  Configuration  Management 

18.  Requirements  Management 

19.  Risk  Management 

20.  Technical  Data  Management 

21.  Interface  Management 

22.  Software  Engineering 

23.  Acquisition 

24.  Systems  Engineering  Leadership 

25.  System  of  Systems 

Professional  (4) 

26.  Communications 

27.  Problem  Solving 

28.  Strategic  Thinking 

29.  Professional  Ethics 

Table  2:  SPRDE-SE/PSE  Competencies  Addressed  in  RT-19A 
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The  universities  that  offered  capstone  courses  were  required  to  address  one  or  more  of  five 
DoD  problem  areas  and  to  produce  an  actual  product,  prototype,  or  other  artifact  to 
demonstrate  their  learning.  Below  is  a  list  of  the  problem  areas  for  RT-19A.  These  were  identical 
to  the  problem  areas  for  RT-19  but  with  the  addition  of  a  problem  area  referring  to  assistive 
technologies. 


DoD  Problem  Areas 

1.  Low-cost,  low-power  computers  leveraging  open-source  technologies  and  advanced 
security  to  support  sustainable,  secure  collaboration; 

Portable,  renewable  power  generation,  storage,  and  distribution  to  support  sustained 
operations  in  austere  environments  and  reduce  dependency  on  carbon-based  energy 
sources;  Portable,  low-power  water  purification; 

2.  An  expeditionary  assistance  kit  around  low-cost,  efficient,  and  sustainable  prototypes 
such  as  solar  cookers,  small  and  transportable  shelters,  deployable  information  and 
communication  technologies,  water  purifiers,  and  renewable  energies.  These  materials 
would  be  packaged  in  mission-specific  HA/DR  kits  for  partner  nation  use; 

3.  Develop  modular,  scalable,  expeditionary  housing  systems  that  possess  "green"  electric 
power  and  water  generation,  waste  and  wastewater  disposal,  hygiene,  and  food  service 
capabilities.  Systems  should  be  designed  to  blend  in  to  natural/native  surroundings  and 
with  minimal  footprint; 

4.  Continued  investigation  and  exploration  into  the  realm  of  the  possible  with  respect  to 
"Immersive"  training  technologies.  Objective  is  to  flood  the  training  audience 
environment  with  the  same  STIMULI  that  one  would  experience  during  actual  mission 
execution.  Where  possible  full  sensory  overload  is  desired  much  the  same  as  experienced 
in  combat.  Specific  S&T  areas  for  development 

Virtual  Human.  Successful  modeling  of  emotions,  speech  patterns,  cultural 
behaviors,  dialogue  and  gestures. 

Universal  Language  Model.  The  ability  for  trainees  to  seamlessly  converse  with  the 
Virtual  Human. 

Virtual  Character  Grab  Controls.  The  ability  for  exercise  controllers  to  assume 
control  of  virtual  characters. 

Automated  Programming.  Cognitive  learning  models  and  the  ability  for  exercise 
controllers  to  adjust  virtual/live  simulations. 

Low  Cost  wireless  personnel  sensors. 

Sensors  (i.e.,  lightweight  vests)  that  facilitate  physical  stimuli  (i.e.,  wounds,  shots)  to 
trainees. 

5.  Assistive  technologies  for  wounded  warriors,  including  but  not  limited  to  application 
of  haptic  research,  augmented  reality,  research  on  traumatic  brain  injury,  bio-medical 
advances,  hybrid  assistive  approaches  (e.g.,  human-  machine  interfaces)  and  other 
leading-  edge  technologies  to  facilitate  rehabilitation  and  contribute  positively  to 
wounded  warrior  quality  of  life. 


Table  3:  DoD  Problem  Areas 
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Systems  Engineering  Capstone  RT  19  Promising  Practices 

1.  Fall  semester  tools/techniques/approaches  systems  engineering  theory  course, 
followed  by  spring  semester  design  project  course.  Fall  course  should  present 
balance  of  "traditional"  systems  engineering  approaches  with  automated  tools/ 
models/  simulation  techniques. 

2.  Creative  imposition  of  technical,  budget,  and  schedule  constraints  by  faculty  to 
model  "real  world." 

3.  Use  of  Systems  Engineering  doctoral  students  as  project  advisors. 

4.  Cross-disciplinary  student  teams. 

5.  Regular,  direct  involvement  of  mentors  with  student  project  teams--  e.g., 
significant  meetings  twice  monthly  with  "on-call"  consultations  between  meetings. 

6.  Creative  use  of  mentors  from  defense  prime  contractors. 

7.  Structured  design  reviews  with  DoD  and  industry  mentors  serving  as  reviewers. 

8.  Civilian  schools  to  establish  relationships  with  nearby  DoD  commands  and  facilities. 

9.  For  civilian  institutions  that  have  on-campus  ROTC  units,  established  relationships 
with  ROTC  units  for  requirements  analysis,  use  case  testing,  and  solution  viability. 


Table  4:  RT  19  Promising  Practices 

In  addition,  the  participating  Pis  were  asked  to  incorporate  as  many  of  the  nine  Promising 
Practices  identified  in  the  previous  year  as  feasible  in  their  courses.  These  are  listed  in  Table  4. 

Finally,  as  noted  above,  the  goal  of  increasing  the  number  of  schools  offering  systems 
engineering  capstone  courses  was  approached  by  developing  partnerships  between  RT-19 
participants  and  non-systems  engineering  schools.  As  a  result,  three  civilian  schools  (Auburn, 
Southern  Methodist  University,  and  University  of  Virginia)  and  two  service  academies  (Coast 
Guard  Academy  and  Naval  Academy)  from  RT-19  created  partnerships  with  six  new  schools 
(Connecticut  College,  Smith  College,  Sweet  Briar,  Tuskegee  University,  University  of  Flawaii 
Manoa,  and  University  of  Rhode  Island).  This  effort  will  be  discussed  in  more  detail  below. 


1.3  RESEARCH  QUESTIONS  AND  METHODS 

The  key  research  questions  this  program  was  designed  to  address  are: 

(1)  What  are  institutional  challenges  and  successes  in  the  adoption  of  core  elements  of 
successful  systems  engineering  capstone  projects? 

(2)  What  are  the  contexts  and  program  characteristics  leading  to  highly  successful  student 
team-developed  products  and  artifacts  that  respond  to  authentic  Department  of 
Defense  (DoD)  problem  areas? 

Each  RT-19A  lead  school  and  some  partner  schools  administered  two  types  of  assessment  to 
their  students: 

•  Customized  pre-/post  assessments  that  were  targeted  to  their  own  course  learning 
objectives.  Assessments  were  typically  developed  by  the  course  instructors  and  related 
to  specific  course  content,  ranging  from  multiple  choice  response  tests,  to  a 
performance-based  assessment,  to  other  types  of  authentic  assessments. 
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•  A  common  student  assessment  developed  by  the  research  team  and  administered  in 
survey  format  at  the  beginning  and  end  of  the  academic  year  that: 

o  Gauged  changes  in  student  involvement  in,  and  understanding  of,  the  systems 
engineering  design  process,  including  requirements  analysis,  project 
management,  and  testing  phases;  system  level  trade-offs;  and  the  nature  and 
type  of  client  and  mentor  interactions 

o  Gauged  changes  in  student  interest  in  systems  engineering  study  and  systems 
engineering  careers,  including  DoD  systems  engineering  careers 
o  Collected  demographics,  including  gender,  race/ethnicity,  major,  class  year  and 
status,  prior  experience  with  systems  engineering,  etc.  from  participating 
students 

In  addition,  faculty  at  each  participating  institution  developed  customized  assessments  that 
were  unique  to  their  courses  using  diverse  instruments  such  as  competency  rubrics,  student 
presentations  for  design  reviews,  peer  reviews,  and  team  reports. 

In  addition  to  analyzing  the  results  of  the  student-  and  faculty-level  assessments  and  surveys, 
the  following  report  includes  case  studies  of  partnerships,  describing  the  best  practices,  models, 
approaches,  and  conditions,  as  well  as  the  ineffective  practices  and  unresolved  challenges. 


1.4  TIMELINE 

The  program  was  implemented  in  three  sequential  phases  over  an  18-month  period: 

During  Phase  1/Planning  and  Startup  (April  1,  2011-June  30,  2011),  the  research  team,  with 
participation  from  the  sponsor  agency,  developed  the  requirements  and  specifications,  timeline, 
and  funding  limits  for  the  systems  engineering  capstone  courses;  developed  the  research  design 
and  project  evaluation  plan;  developed  and  issued  the  request  for  proposals  and  selection 
process  (through  an  independent  review  team  and  rubric)  for  selecting  participating  schools; 
and  selected  six  systems  engineering  member  schools  and  four  service  academies  with  systems 
engineering  or  general  engineering  programs  that  would  participate  in  the  project.  As  noted 
above,  five  of  those  schools  (hereafter  "lead  schools")  recruited  six  non-systems  engineering 
partner  schools. 

During  Phase  2/Development  and  Implementation  (July  1,  2011-June  30,  2012),  participating 
schools  that  would  offer  capstone  courses  recruited  student  participants;  developed  and 
organized  course  materials;  coordinated  interactions  between  students,  mentors,  and  clients; 
planned  assessments;  delivered  systems  engineering  instruction  to  student  teams;  and 
participated  in  recommended  student  competitions  and  conferences  (Spring  2012).  As  we  will 
see  below,  lead  schools  with  partners  managed  their  capstone  course  and  prototype 
development  in  different  ways.  Finally,  Pis  from  all  the  participating  schools  submitted  an 
interim  and  final  survey  that  asked  about  the  scaling  process,  the  challenges  to  sustainability, 
and  the  reasons  behind  the  success  (or  lack  thereof)  demonstrated  by  the  student  prototypes. 

During  Phase  3/Analysis,  Recommendations  &  Dissemination  (July  1,  2012  -  September  30, 
2012),  the  research  team  analyzed  results  from  all  participating  schools  and  integrated  them 
into  a  single  set  of  findings  about  the  effectiveness  of  the  programs  using  a  variety  of  metrics: 
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•  Institutional  infrastructure  and  institutionalization 

•  Effectiveness  of  course  structure,  materials,  and  external  inputs  (mentors  and  clients) 

•  Success  of  student  projects 

•  Student  learning  of  systems  engineering  skills  and  competencies 

•  Partnerships  as  a  means  of  scaling  up 
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2.0  THE  CAPSTONE  EXPERIENCE 


This  section  of  the  report  will  look  at  the  success  of  different  aspects  of  the  capstone 
experience.  It  is  based  on  a  series  of  surveys  given  to  the  three  groups  that  participated  in  RT- 
19A:  primary  investigators  (instructional,  as  well  as  supporting  administrative  or  advisory 
faculty);  undergraduate  and  graduate  students  who  were  enrolled  in  the  capstone  courses;  and 
mentors  (DoD  and  industry).  Primary  investigators  (Pis)  were  asked  to  respond  to  an  interim  and 
final  survey.  Although  16  schools  were  involved  in  RT-19A,  13  Pis  responded  to  the  interim 
survey  and  9  to  the  year-end  survey.  The  three  that  did  not  respond  to  the  interim  survey  were 
Tuskegee,  University  of  Rhode  Island,  and  Naval  Postgraduate  School,  while  the  seven  that  did 
not  respond  to  the  year-end  survey  included  five  of  the  six  partner  schools  along  with  the  Air 
Force  Academy  and  the  Naval  Postgraduate  School.  As  noted  above,  Tuskegee  University  did  not 
create  a  capstone  course  or  enroll  students  but  instead  monitored  Auburn's  course  so  the 
survey  did  not  apply  to  them,  while  the  Air  Force  Academy  had  not  finished  by  the  time  the  final 
survey  went  out  and  the  Naval  Postgraduate  School  operated  on  an  entirely  different  schedule 
from  the  other  institutions.  The  only  partner  school  PI  who  answered  the  final  survey  was  from 
the  University  of  Hawaii-Manoa,  one  of  the  more  successful  partnerships  (see  below). 

Students  were  asked  to  respond  to  a  baseline  survey  administered  in  September  2011  and  year- 
end  survey  administered  in  May  2012.  The  response  differed  by  institution,  with  more  complete 
data  presented  in  the  appropriate  section.  Mentors  were  asked  to  respond  to  a  survey  sent 
them  in  May  2012. 


2.1  CAPSTONE  COURSE  ORGANIZATION 

One  of  the  recommended  promising  practices  was  to  implement  the  capstone  course  over  two 
semesters,  with  the  first  semester  focusing  on  theory  and  the  second  on  design.  With  the 
exception  of  Tuskegee,  all  of  the  RT-19A  schools  implemented  the  systems  engineering 
capstone  course  experience  over  two  semesters.  However,  only  the  Naval  Academy  adopted  the 
practice  of  having  formal  lectures  on  systems  engineering  theory  during  the  fall  semester  and 
confining  prototype  design  to  the  spring  semester,  although  even  here,  the  students  designed 
paper  prototypes  in  the  fall.  Most  of  the  other  schools  reported  that  students  worked  on  a 
combination  of  theory-related  lecture  and  prototype  development  or  systems  testing  during  the 
fall,  although  at  University  of  Virginia  students  did  not  receive  formal  lectures  but  learned 
systems  engineering  concepts  in  a  "just-in-time"  manner  as  they  developed  their  prototypes. 

The  nine-month  academic  calendar,  delays  in  materials  acquisition,  technical  problems  with 
components  or  prototypes,  and  other  issues  were  cited  by  the  remaining  Pis  as  reasons  for 
beginning  the  design  process  as  early  as  possible. 

One  promising  practice  was  to  have  Pis  impose  technical,  budget,  and  schedule  constraints  on 
the  students'  projects  in  order  to  model  the  real  world.  All  the  Pis  reported  that  they  had  done 
this,  and  in  fact  it  was  built  into  the  semester  structure  and  each  school's  existing  budget 
constraints. 

The  extent  to  which  students  developed  prototypes  varied  from  school  to  school  (see  Table  5). 
At  three  schools  (Coast  Guard  Academy,  Naval  Academy,  Southern  Methodist  University),  the 
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goal  of  the  capstone  courses  was  to  develop  functional  physical  prototypes.  At  Stevens,  the  PI 
noted  that  the  prior  year's  prototypes  had  been  conceptual  in  nature,  this  year  the  students  had 
focused  on  "data  acquisition  and  model  validation."  Similarly,  students  in  Missouri  University  of 
Science  and  Technology's  Physical  Artifact  capstone  course  worked  on  validating  the  designs 
created  by  capstone  students  in  RT-19.  Student  teams  at  three  schools  worked  on  development 
of  two  or  more  different  prototypes.  At  Smith  College  and  University  of  Hawaii  Manoa,  students 
worked  on  subsystems  for  their  partner  schools. 


School 

Prototype 

Air  Force  Academy 

A.  Small  scale,  low  voltage,  battery  management  and 
charging  system 

B.  Small  scale  model  of  the  power  plant  and  vehicle 
providing  proof  of  concept 

Auburn 

Lightweight,  portable  unmanned  aerial  vehicle  (UAV) 
launched  by  a  soldier  to  reconnoiter  a  hostile 
environment.  Teams  built  a  rotary  wing  UAV  and 
"lighter  than  air"  UAV 

Coast  Guard  Academy 

A.  Shipboard  wastewater  treatment  system 
development  for  Coast  Guard  cutters  (includes 
development  of  membrane-bioreactor  for  treating 
shipboard  gray  water  and  pollutant  removal) 

B.  Natural  gas  engine  conversion 

C.  Autonomous  sailing  vessels 

Connecticut  College 

Small  sailing  robots  that  can  operate  autonomously  in 
navigation  and  communicate  with  each  other  for 
coordinated  operations 

Military  Academy 

Cockpit/Crew  Station  of  the  Future  (2035)  used  as  a 
simulator  to  train  pilots. 

Missouri  University  of  Science  and 
Technology 

Immersive  training  vests  with  position  reporting  and 
vibrators 

Naval  Academy 

Fully  functional,  independently  powered  (e.g. 
renewable  power  source)  water  purification  system 
capable  of  supporting  at  least  80  people  from  multiple 
water  sources 

Smith  College 

Water  purification  system  -  specifically,  power  sub¬ 
system  in  partnership  with  Naval  Academy 

Southern  Methodist  University 

Interactive,  immersive  training  environment  with 
human  gesture  tracking  and  facial  emotion  capture 
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Stevens 

Prepositioned  Expeditionary  Assistance  Kit/Green 
Housing  -  Shelter,  grey  water  recycling  system,  tied 
into  on-campus  mechanical  tri-generation  systems, 
alternative  energy  (wind  belt)  and  overall  simulator 

Sweet  Briar 

Immersive  technology  to  alleviate  phantom  limb  pain 

University  of  Hawaii 

Distributed  systems  assurance  processes  and  methods 
in  partnership  with  Southern  Methodist  University 

University  of  Virginia 

Rapid  Adaptive  Needs  Assessment  (water  sampling)  kit 
of  physical  sensors  and  communications  equipment  in 
waterproof  container  with  anchoring  systems 

Virtual  environment,  hardware  to  sense  location  and 
grip  of  user's  hand,  and  structure  designed  to  project 
virtual  environment  onto  a  tabletop 

Table  5:  Types  of  Prototypes  Developed  in  RT-19A 


2.2  SYSTEMS  ENGINEERING  COMPETENCIES  AND  DEFINITIONS 

The  SPRDE-SE/PSE  Competency  Model  guided  faculty  design  of  course  foci  over  the  two 
semester  long  capstone  experience.  Competencies  that  were  most  frequently  listed  as  course 
foci  included  Problem  Solving  (84.6%),  Stakeholder  Requirements  Definition  (69.2%), 
Requirements  Analysis  (69.2%),  Verification  (69.2%),  and  Communication  (61.5%).  The 
competencies  among  the  most  infrequently  reported  as  course  foci  were  Technical  Assessment, 
Configuration,  and  Acquisition. 

Pis  were  split  between  those  who  presented  students  with  formal  definitions  of  systems 
engineering  and  those  who  preferred  more  experiential  interpretations.  Auburn,  Stevens,  Air 
Force  Academy,  University  of  Virginia,  and  Naval  Academy  all  reported  using  the  definition  of 
systems  engineering  from  the  International  Council  of  Systems  Engineering  (INCOSE)  to  help 
students  clarify  the  main  concepts  and  relations  of  a  complex  discipline: 

Systems  engineering  is  an  interdisciplinary  approach  and  means  to  enable  the 
realization  of  successful  systems.  It  focuses  on  defining  customer  needs  and  required 
functionality  early  in  the  development  cycle,  documenting  requirements,  and  then 
proceeding  with  design  synthesis  and  system  validation  while  considering  the  complete 
problem:  operations,  cost  and  schedule,  performance,  training  and  support,  test, 
manufacturing,  and  disposal.  Systems  engineering  considers  both  the  business  and  the 
technical  needs  of  all  customers  with  the  goal  of  providing  a  quality  product  that  meets 
the  user  needs. 

At  Southern  Methodist  University,  the  PI  reported  that  students  benefited  from  first  learning 
practical  design  skills  ("interface  management,  iterative  systems  development,  integration, 
etc.")  before  combining  those  skills  into  a  theoretical  understanding  of  systems  engineering: 
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Based  on  RT-19  experiences,  we  only  confused  students  by  introducing  the  general  and 
abstract  systems  engineering  concepts  that  early.  Thus  in  Fall  2011  Senior  Design  I,  we 
educated  students  on  the  skill  set  of  systems  engineering  including  how  to  perform 
iterative  system  development,  system  integration  and  interface  management,  risk 
identification  and  management.  In  addition,  students  learned,  understood  and  applied 
the  above  skills  along  with  the  project  design  and  prototyping.  Then  in  Spring  2012,  we 
will  arrange  a  joint  systems  engineering  lecture  to  define  systems  engineering  and 
discuss  why  we  need  systems  engineering,  etc. 

Finally,  other  Pis  defined  systems  engineering  for  their  students  more  informally,  stressing  the 
"big  picture  of  systems";  the  role  of  the  "multi-  or  inter-disciplinarity  of  engineering  disciplines" 
in  the  development  and  maintenance  of  systems;  "project  management";  and  the  need  for 
"designing  optimal  solutions  for  the  client  while  analyzing  risk." 


2.3  FACULTY  INVOLVEMENT 

Fifty-one  faculty  members  participated  in  the  development,  delivery,  or  assessment  of  RT-19A 
courses,  almost  the  same  number  as  participated  in  RT-19  (50).  This  year's  faculty  came  from 
Industrial,  Electrical,  Civil,  Mechanical,  Systems,  Software,  Ocean  Engineering,  and  Computer 
Science.  The  highest  percentage  came  from  Mechanical  Engineering,  followed  by  Systems 
Engineering.  Eight  schools  included  faculty  participants  from  more  than  one  engineering 
discipline. 


Figure  1  shows  the  percentages  of  all  faculty  members  in  the  project: 
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Figure  1:  RT-19A  Participating  Faculty  by  Discipline 
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2.4  GRADUATE  STUDENT  INVOLVEMENT 

Another  RT-19  Promising  Practice  was  to  have  Systems  Engineering  doctoral  students  act  as 
project  advisors  to  students  enrolled  in  the  capstone  courses.  This  was  a  much  more  prevalent 
practice  this  year  than  last  year,  with  eleven  graduate  students  at  seven  different  institutions 
acting  as  project  managers,  teaching  assistants,  and/or  mentors  to  RT-19A  teams.  These 
students  helped  teams  manage  risk,  balance  workload  across  team  members,  and  coordinate 
deliverables  and  project  artifacts  in  a  timely  manner.  Thus  at  Southern  Methodist  University  the 
student  teaching  assistant  monitored  students'  progress  through  weekly  project  status  reports 
and  surveys.  Systems  engineering  doctoral  students  at  Missouri  University  of  Science  and 
Technology  facilitated  weekly  WebEx  meetings  between  capstone  students,  the  faculty  advisor, 
and  industry  mentors.  The  graduate  student  project  manager  at  Stevens  was  a  previous 
participant  in  the  RT-19  effort  who  served  as  a  knowledge  resource  for  the  RT-19A  teams 
working  on  a  new  iteration  of  the  previous  year's  problem  area.  At  Auburn,  the  graduate 
student  assistant  worked  to  ensure  that  systems  engineering  concepts  were  employed  and  that 
outside  help  was  sought  when  appropriate.  Finally,  at  Air  Force  Academy  and  University  of 
Virginia,  graduate  students  acted  as  occasional  mentors  who  responded  to  technical  inquiries. 


2.5  STUDENT  RECRUITMENT 

Faculty  employed  multiple  strategies  to  recruit  students  to  the  capstone  courses,  with  most 
considering  face-to-face  recruitment  to  be  the  most  effective  strategy. 

At  Stevens  and  Auburn,  word  of  mouth  from  students  in  the  previous  year  had  the  greatest 
impact  on  recruitment.  Pis  at  University  of  Virginia  reported  that  conversations  with  faculty 
from  other  departments  were  helpful  while  Pis  at  Southern  Methodist  University  said  that  they 
had  help  from  senior  design  faculty.  The  graph  below  shows  the  methods  most  frequently  used, 
with  most  schools  using  more  than  one: 
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Figure  2:  Methods  of  Recruiting  Students 
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2.6  DOD  PROBLEM  AREAS  ADDRESSED 

Each  of  the  universities,  including  lead  and  partner  schools,  chose  one  or  more  of  five  DoD 
problem  areas.  Three  schools  addressed  multiple  problem  areas.  Problem  areas  2  and  4  were 
the  most  frequently  chosen,  while  problem  areas  3  and  5  were  least  represented.  Figure  3 
shows  the  percentage  of  schools  choosing  each  area,  while  Table  6  shows  the  problem  areas 
addressed  at  each  school. 


5:  Assistive  technologies  for  wounded 

warriors 

4:  Immersive  training  technologies 

3:  Expeditionary  housing  systems 

2:  Expeditionary  assistance  kit 

1:  Low-cost,  low-power  computers 
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Figure  3:  DoD  Problem  Areas  Addressed 


RT-19A  School 

Problem  Area 

Air  Force  Academy 

2 

Auburn 

1 

Coast  Guard  Academy 

1,2,3 

Connecticut  College 

1,2 

Military  Academy 

4 

Missouri  University  of  Science  and  Technology 

4 

Naval  Academy 

2 

Smith  College 

2 

Southern  Methodist  University 

4 

Stevens 

3 
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Sweet  Briar 

4,5 

University  of  Hawaii 

4 

University  of  Virginia 

2,4,5 

Table  6:  Problem  Areas  by  School 

The  Pis  listed  student  interest,  faculty  interest,  and  faculty  subject  matter  expertise  as  the  most 
important  reasons  for  selecting  a  problem  area.  At  several  schools  (Missouri  University  of 
Science  and  Technology,  Stevens,  Southern  Methodist  University,  and  University  of  Virginia), 
students  worked  on  problem  areas  carried  over  from  the  previous  year.  Two  schools  reported 
that  they  selected  the  problem  area  based  on  client  needs  (Military  Academy,  Southern 
Methodist  University).  At  two  partner  schools  (Connecticut  College,  University  of  Hawaii 
Manoa),  the  lead  school  decided  on  the  problem  area.  Figure  5  shows  the  percentage  of  Pis  who 
listed  each  reason. 


Hardware  capability 
Partner  selected 
Client  needs 
Extension  of  last  year's  project 
Faculty  subject  matter  expertise 
Faculty  research  interest 
Student  research  interest 


Figure  4:  PI  Reasons  for  Selecting  Problem  Areas 


2.7  ROTC  PARTICIPATION  AND  RELATIONSHIPS  WITH  NEARBY  DOD  FACILITIES 

Two  other  recommendations  from  RT-19  were  that  schools  collaborate  with  on-site  or  nearby 
ROTC  units  for  "requirements  analysis,  use  case  testing,  and  solution  viability"  and  that  civilian 
schools  establish  relationships  with  nearby  DoD  commands  or  facilities.  Only  two  schools 
reported  that  they  had  an  existing  ROTC  unit.  At  the  Naval  Academy,  the  PI  interpreted  the 
definition  of  ROTC  broadly  and  called  the  entire  school  "essentially  an  on-campus  ROTC  unit." 
Auburn  facilitated  such  a  relationship  by  using  a  sergeant  in  the  local  Army  ROTC  unit  with  UAV 
deployment  experience  "to  validate  the  description  of  the  functional  requirements"  of  the 
student  prototype  during  the  spring  semester.  As  for  establishing  relationships  with  nearby 
command  facilities,  two  civilian  schools  (Smith  and  Connecticut  College)  established  such 
relationships  by  partnering  with  military  schools,  but  none  of  the  other  civilian  schools 
established  new  relationships. 
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3.0  STUDENT  PARTICIPATION 


3.1  STUDENT  PARTICIPATION  IN  CAPSTONE  COURSES 

A  total  of  fifteen  schools  in  RT-19A  enrolled  students  in  a  systems  engineering  capstone  course, 
compared  to  fourteen  schools  in  RT-19.  Figure  5,  using  data  from  the  interim  faculty  surveys, 
compares  the  participation  rate  in  RT-19  with  that  for  RT-19A.  It  should  be  noted  that  the  total 
for  the  spring  semester  was  a  prediction  made  before  the  semester  began.  It  was  larger  than  the 
total  for  the  fall  semester  primarily  because  of  increased  participation  at  two  of  the  service 
academies  (Coast  Guard  Academy  and  Naval  Academy). 


Figure  5:  Student  Participation,  Fall  and  Spring  Semesters,  RT-19  and  RT-19A  Compared 

However,  the  number  of  students  who  enrolled  in  the  capstone  courses  was  higher  than  the 
number  of  students  who  worked  specifically  on  DoD  problem  areas.  Thus  the  Pis  reported  that 
198  students  were  expected  to  work  on  DoD  problem  areas  during  the  spring  semester,  or 
64.7%  of  the  total  number  of  students  enrolled  in  the  course. 

The  Pis  reported  that  team  sizes  ranged  from  2  -15,  with  teams  of  three,  four,  and  five  students 
most  frequent.  However,  there  was  a  great  deal  of  variety  in  team  size  and  number  of  teams  at 
each  school.  For  example,  the  Naval  Academy  had  two  of  the  largest  teams,  with  13  and  15 
students  on  each,  while  the  Coast  Guard  Academy  had  the  greatest  number  of  teams  (10)  but 
these  had  only  four  students  per  team. 
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3.2  DEMOGRAPHICS  OF  STUDENT  PARTICIPANTS 

This  section  will  provide  an  overview  of  the  demographic  characteristics  of  the  students 
engaged  in  the  RT-19A  capstone  courses.  It  is  based  on  an  analysis  of  student  background 
surveys,  which  were  received  from  12  of  the  16  participating  schools  with  students.  In  Table  7 
those  schools  with  no  student  responses  are  highlighted  in  gray.  As  noted  above,  Connecticut 
College  and  University  of  Rhode  Island  joined  their  projects  late,  while  Tuskegee  had  no 
students  and  Naval  Postgraduate  School  is  not  included  because  it  was  on  a  different  schedule 
from  the  other  participating  schools. 


RT-19A  Lead  School 

Partner  School 

Air  Force  Academy 

Auburn 

Tuskegee  University 

Coast  Guard  Academy 

Connecticut  College 

University  of  Rhode  Island 
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Missouri  University  of  Science  &  Technology 

Military  Academy 

Naval  Academy 

Smith  College 

Naval  Postgraduate  School 

Southern  Methodist  University 

University  of  Hawaii  Manoa 

Stevens 

University  of  Virginia 

Sweet  Briar 

Table  7:  Schools  with  Students  Submitting  Background  Surveys 

A  total  of  196  students  returned  the  baseline  survey,  or  68%  of  the  number  of  students  reported 
by  the  Pis  to  have  been  enrolled  in  the  fall  semester  capstone  course.  Table  8  shows  the  number 
of  baseline  surveys  returned  from  each  school. 


Baseline 

surveys 

Air  Force  Academy 

31 

Auburn 

31 

Coast  Guard  Academy 

26 

Military  Academy 

4 

Missouri  University  of  Science  and  Technology 

19 

Naval  Academy 

28 

Smith  College 

4 

Southern  Methodist  University 

8 

Stevens 

20 

Sweet  Briar 

4 

University  of  Hawaii 

4 

University  of  Virginia 

17 

Total 

196 

Table  8:  Baseline  Survey  Responses 

Based  on  these  responses,  the  following  sections  will  discuss: 

•  Survey  participation  rate 

•  Academic  status  and  class  year 

•  Major 


RT  19a  Final  Technical  Report,  SERC-TR-019-2,  September  30,  2012 


27 


UNCLASSIFIED 


•  Gender  and  Ethnicity 

•  Experience  with  general  engineering 

•  Experience  with  systems  engineering 

•  Systems  engineering  career  interest 

Comparisons  between  RT-19  and  RT-19A  will  be  made  where  relevant.  Because  the 
demographic  student  data  is  based  on  a  subset  of  all  the  participants,  any  generalizing  from  the 
results  must  be  done  with  caution. 


3.3  ACADEMIC  STATUS  AND  CLASS  YEAR 

It  appears  from  the  survey  responses  that  RT-19A  involved  more  undergraduates  and  far  fewer 
graduate  students  (as  students,  not  mentors)  than  RT-19  (see  Figure  7). 


Undergraduate  Graduate  Postgraduate 

fc  RT-19  ■  RT-19  A 


Figure  7:  Number  of  Undergraduate  and  Graduate  Students  in  RT-19  and  RT-19A 

In  addition,  while  for  RT-19,  several  schools  had  a  mixture  of  undergraduates  and  graduates  in  a 
single  class,  for  RT-19A,  students  from  only  one  school  (Auburn)  reported  a  mix— in  this  case,  a 
class  of  31  that  was  two-thirds  graduate  students  and  one-third  undergraduates.  Missouri 
Institute  of  Science  and  Technology  had  entirely  graduate  students  while  all  the  others  had 
entirely  undergraduates. 

Most  of  the  undergraduate  respondents  were  seniors,  while  most  of  the  graduate  students 
were  in  their  first  or  second  year  of  graduate  school.  Tables  9  and  10  show  class  status  for  all 
respondents  and  by  individual  school. 
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Frequency 

Percent 

First  year  graduate  student 

19 

9.7 

Second  year  graduate  student 

15 

7.7 

Third  through  fifth  year  graduate  student 

8 

4.1 

Undergraduate  Junior 

1 

0.5 

Undergraduate  Senior 

150 

76.5 

No  response 

3 

1.5 

Total 

196 

100.0 

Table  9:  Survey  Respondents  by  Class  Year 


Class  Status 

Total 

Grad 

Postgrad 

Undergrad 

Air  Force  Academy 

0 

0 

31 

31 

Auburn 

21 

0 

10 

31 

Coast  Guard  Academy 

0 

0 

26 

26 

Military  Academy 

0 

0 

4 

4 

Missouri  University  of  Science 
and  Technology 

18 

1 

0 

19 

Naval  Academy 

0 

0 

28 

28 

Smith  College 

0 

0 

4 

4 

Southern  Methodist  University 

0 

0 

8 

8 

Stevens 

0 

0 

20 

20 

Sweet  Briar 

0 

0 

4 

4 

University  of  Hawaii 

0 

0 

4 

4 

University  of  Virginia 

0 

0 

17 

17 

Total 

39 

1 

156 

196 

Table  10:  Survey  Respondents  by  School 
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3.4  STUDENT  MAJORS 

The  most  common  major  overall  was  Systems  Engineering,  followed  by  Mechanical  and 
Electrical  Engineering  (see  Figure  8).  About  30%  of  those  returning  surveys  were  Systems 
Engineering  majors,  distributed  among  seven  of  the  twelve  schools.  Students  majoring  in 
Mechanical  Engineering  were  distributed  across  three  schools  while  students  majoring  in 
Electrical  Engineering  were  distributed  across  four  schools.  Majors  represented  by  only  one 
student  included  Accounting  and  Finance,  Biomedical  Engineering,  Environmental  Science,  and 
Computer  Science  Engineering. 


Systems  Engineering 
Mechanical  Engineering 
Electrical  Engineering 
Industrial  Engineering 
Software  Engineering 


31.5% 

30.8% 


23.5% 

28.4% 


14.1% 

12.8% 


3.4% 

8.1% 


8.1% 

7.1% 


Civil  Engineering 


Computer  Science 


Engineering  Management 


Biomedical  Engineering 


2.0% 

1.4% 


P 

P 


11.4% 

7.1% 


5.4% 

3.8% 


,  0.7% 

0.5% 
[ _ 


■  RT-19A 
B  RT-19 


0.0%  20.0%  40.0%  60.0%  80.0%  100.0% 


Figure  8:  Student  Majors,  RT-19  and  RT-19A  Compared 

Another  recommendation  from  RT-19  was  that  students  form  cross-disciplinary  teams.  In  RT- 
19A,  students  at  nine  of  the  schools  that  had  students  responding  to  the  baseline  survey  came 
from  two  or  more  engineering  disciplines,  while  students  at  three  schools  came  from  one 
engineering  discipline.  For  two  of  these  three  (Smith  College  and  Sweet  Briar),  this  was  the  only 
major  available.  Over  three-quarters  of  the  students  who  answered  "Other"  described  their 
major  as  General  Engineering,  while  most  of  the  rest  were  in  non-engineering  majors  (see  Table 
11).  Table  12  shows  engineering  majors  by  school. 
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General  Engineering 

20 

77.0 

Information  Technology  Management 

3 

11.6 

Accounting  and  Finance 

1 

3.8 

Environmental  Science 

1 

3.8 

Computer  Science  Engineering 

1 

3.8 

Total 

26 

100.0 

Table  11:  "Other"  Majors 


School 

Disciplines 

Air  Force  Academy 

Computer  Engineering,  Electrical 

Engineering,  Engineering  Management, 
Systems  Engineering 

Auburn 

Systems  Engineering,  Mechanical 

Engineering,  Electrical  Engineering 

Coast  Guard  Academy 

Civil  Engineering,  Electrical  Engineering, 
Mechanical  Engineering 

Military  Academy 

Systems  Engineering 

Missouri  University  of  Science  & 
Technology 

Electrical  Engineering,  Computer 

Engineering,  Mechanical  Engineering, 
Industrial  Engineering,  Systems  Engineering 

Naval  Academy 

General  Engineering,  Systems  Engineering 
(Multidisciplinary  major) 

Smith  College 

General  Engineering 

Southern  Methodist  University 

Computer  Science  (software  focus), 

Computer  Engineering  (hardware  focus) 

Stevens 

Mechanical  Engineering,  Electrical 

Engineering,  Civil  Engineering,  Engineering 
Management,  Computer  Engineering, 

Systems  Engineering 

Sweet  Briar 

General  Engineering 

University  of  Hawaii  Manoa 

MIS,  Finance,  Accounting,  Management 
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University  of  Virginia 

Systems  Engineering,  Engineering 

Management,  Biomedical  Engineering, 

Computer  Engineering 

Table  12:  Engineering  Majors  by  School 


3.5  GENDER  AND  ETHNICITY 

The  student  population  that  returned  surveys  was  over  three-quarters  male,  approximately  the 
same  as  last  year,  and  about  20%  female  compared  to  17%  last  year.  This  increase  was  in  part 
due  to  the  participation  of  two  women's  college,  Sweet  Briar  and  Smith,  but  the  overall  balance 
is  in  line  with  engineering  schools  nationally. 


Gender 

RT-19A 

Frequency 

RT-19A 

Percent 

RT-19 

Percent 

Male 

154 

78.6 

76.8 

Female 

40 

20.4 

17.4 

No  response 

2 

1.0 

6.8 

Total 

196 

100.0 

100.0 

Table  13:  Gender,  RT-19  and  RT-19A  Compared 


The  ethnicity  of  the  responding  students  was  only  slightly  different  from  the  ethnicity  reported 
last  year.  Over  two-thirds  (68%)  of  the  students  in  RT-19A  reported  their  ethnicity  as  White, 
slightly  more  than  the  64%  for  RT-19.  This  was  followed  by  Asian  and  African-American/black 
students,  with  the  percentage  of  the  latter  slightly  higher  than  the  6.8%  last  year. 


Ethnicity 

RT-19A 

Frequency 

RT-19A 

Percent 

RT-19 

Percent 

White 

133 

67.9 

64.3 

Asian 

21 

10.7 

11.4 

Black  or  African  American 

21 

10.7 

6.8 

Hispanic/Latino 

5 

2.6 

5.3 

Native  Hawaiian  or  Other  Pacific  Islander 

2 

1.0 

0.01 

American  Indian/Alaska  Native 

3 

1.5 

0.01 

No  response 

11 

5.6 

11.4 

Total 

196 

100.0 

100.0 

Table  14:  Ethnicity 
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3.6  PRIOR  EXPERIENCE  WITH  ENGINEERING 

All  but  13  (6.6%)  of  the  survey  respondents  reported  having  had  prior  engineering  experience, 
either  through  full-time  employment,  an  internship  or  co-op,  summer  work,  or  a  combination  of 
these.  However,  almost  one-third  did  not  respond  to  this  question,  presumably  because  they 
had  no  experience,  so  the  percentage  with  no  experience  may  have  been  closer  to  40%.  These 
percentages  where  almost  the  same  as  last  year. 


Frequency 

Percent 

Some  engineering  experience 

116 

59.2 

No  engineering  experience 

13 

6.6 

No  response 

67 

34.2 

Total 

196 

100.0 

Table  15:  Prior  Engineering  Experience 

A  higher  percentage  of  the  responding  students  (49%)  reported  having  no  systems  engineering 
experience  this  year  compared  to  last  year,  when  only  40%  reported  such  experience.  As  can  be 
seen  in  Table  16,  the  largest  percentage  of  those  who  reported  this  type  of  experience  had 
gained  it  through  coursework  (28.6%),  followed  by  summer  employment  (10.7%). 


Frequency 

Percent 

System  engineering  experience 

77 

39.3 

Not  sure 

17 

8.7 

No 

96 

49.0 

No  response 

6 

3.0 

Total 

196 

100.0 

Table  16:  Prior  Systems  Engineering  Experience 


3.7  PRIOR  EXPERIENCE  WITH  MULTIDISCIPLINARY  TEAMS 

As  noted  above,  one  promising  practice  was  to  have  students  work  in  multidisciplinary  teams. 
This  year,  only  about  half  (52%)  of  the  students  reported  that  they  had  prior  experience  working 
in  multidisciplinary  teams,  generally  in  an  engineering  course  or  other  academic  context. 
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Frequency 

Percent 

Yes,  in  an  engineering  course 

88 

44.9 

Yes,  in  another  course 

14 

7.1 

Not  sure 

21 

10.8 

No 

73 

37.2 

Total 

196 

100.0 

Table  17:  Prior  Experience  with  Multidisciplinary  Teams 
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4.0  FACULTY  PERCEPTIONS  OF  RT-19A 
SUCCESSES  AND  CHALLENGES 


4.1  PERCEPTIONS  OF  STUDENT  ENGAGEMENT 

When  the  Pis  were  asked  what  had  engaged  the  students  about  their  projects,  they  cited 
interest  in  real-world  problem  areas;  "hands-on"  design  and  teamwork  opportunities; 
experience  with  the  systems  engineering  process;  and  exposure  to  industry  professionals.  These 
were  the  same  attractions  reports  last  year.  Here  are  some  examples  from  the  PI  surveys: 

The  real-world,  tangible  nature  of  the  problem  they  were  working  on.  2.  The  ability  to 
get  lots  of  hands-on  work  building  their  system.  3.  Excitement  about  being  afforded  the 
opportunity  to  complete  a  capstone  project.  4.  Having  2  teams  at  USNA  working  on  the 
same  problem.  Led  to  a  healthy  sense  of  competition. ..The  hands-on  work  on  the 
projects  was  especially  rewarding  for  the  students.  Seeing  their  ideas  come  to  physical 
fruition  and  getting  to  test  them  was  very  rewarding.  Conducting  Integration, 
Verification,  and  Validation  testing  at  each  step  as  they  progressed  from  small  sub¬ 
system  to  overall  working  prototype  really  made  the  Systems  Engineering  process  come 
to  life  for  the  students.  (Naval  Academy) 

Student  development  teams  started  understanding  the  role  of  their  UH  collaborative 
teams  and  tried  to  communicate  with  each  other.  One  team  well  leveraged  the  UH 
system  assurance  team  helped  them  to  develop  the  risk  management  reports  for  their 
design  review  presentations  and  final  design  review.  (Southern  Methodist  University) 

Driving  factors  for  student  engagement  included  that  each  team  worked  on  a  project  in 
an  area  of  relevance  to  their  faculty  advisor,  that  the  teams  were  expected  to  build  and 
test  their  design,  and  that  the  deliverables  for  the  project  were  not  so  frequent  and 
cumbersome  as  to  detract  from  design  and  testing.  (University  of  Virginia) 

Pis  at  two  schools  cited  student  interest  in  designing  prototypes  and  systems  with  applications 
in  sustainable  energy  and  humanitarian  relief  as  important  attractions: 

Students  like  things  that  MOVE,  make  noise,  or  make  lights.  The  off-road  vehicle  appeals 
to  the  MOVE  motivator  but  also  appeals  to  the  growing  sense  of  a  need  for  non-fossil 
fuel  based  vehicles.  (Air  Force  Academy) 

Students  have  an  intrinsic  interest  in  projects  that  involve  sustainability  in  engineering 
and  the  connection  to  disaster  relief  provides  additional  engagement.  The  industry 
mentors  have  further  reinforced  these  themes  through  their  direct  experience. 

(Stevens) 

This  year,  however,  faculty  also  included  interactions  between  students  and  mentors,  faculty 
expertise  carried  over  from  last  year's  project,  competition  among  teams,  making  full  use  of 
partner  teams,  and  the  prestigious  nature  of  the  DoD  projects  as  important  in  engaging 
students.  Tablel8  lists  the  aspects  of  student  engagement  cited  by  faculty  at  each  school. 
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Communication  with  clients  and  mentors 

Air  Force  Academy 

Coast  Guard  Academy 

Military  Academy 

Missouri  University  of  Science  and  Technology 
Southern  Methodist  University 

University  of  Virginia 

Interest  in  real-life  problem 

Auburn 

Naval  Academy 

Smith  College 

Sweet  Briar 

University  of  Virginia 

University  of  Hawaii  Manoa 

Grasp  of  systems  engineering  content 
knowledge 

Air  Force  Academy 

Missouri  University  of  Science  &  Technology 
Southern  Methodist  University 

Smith  College 

University  of  Hawaii  Manoa 

Creation  of  a  physical  prototype  -  "hands- 
on"  activity 

Auburn 

Coast  Guard  Academy 

Naval  Academy 

Weekly  debriefing  and  planning  meetings 
between  Pis  and/or  teaching  assistants 

Auburn 

Missouri  University  of  Science  &  Technology 
University  of  Virginia 

Faculty  technical  and  teaching  experience 
carried  over  from  last  year's  project 

Stevens 

University  of  Virginia 

Collaboration  between  student  teams 
(teams  include  capstone  teams,  internal 
university  collaborations  &  partner  schools) 

Coast  Guard  Academy 

Southern  Methodist  University 

University  of  Virginia 

Utilization  of  subject  matter  expertise 

Sweet  Briar 

University  of  Virginia 

University  of  Hawaii  Manoa 

Experiencing  the  systems  engineering 
process  from  concept  to  testing 

Naval  Academy 

University  of  Virginia 

Prestige  of  DoD  projects 

Southern  Methodist  University 

University  of  Hawaii  Manoa 

Competition  against  other  teams 

Military  Academy 

Naval  Academy 

Work  with  a  faculty  member  from  another 

University  of  Virginia 
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discipline 

Solicitation  of  RT-19A  students 

University  of  Hawaii  Manoa 

Student  team  organization 

Naval  Academy 

Communication  between  Pis/ graduate 
student  advisors  to  students 

Military  Academy 

Assignment  of  Systems  Engineering  PhD 
students  as  project  managers 

Military  Academy 

Increased  professional  and  academic 
networking  opportunities 

Connecticut  College 

Table  18:  Aspects  of  Student  Engagement  by  School 


4.2  PERCEPTIONS  OF  CHALLENGES 

The  Pis  reported  a  number  of  challenges,  several  of  which  were  common  to  many  schools  and 
some  of  which  were  particular  to  only  one  or  two.  Many  of  these  challenges  were  technical 
problems  with  physical  prototypes,  their  components,  and  systems  integration,  although  some 
were  related  to  communication  over  distances. 

In  the  interim  survey,  nearly  half  of  the  Pis  reported  that  students  struggled  with  systems 
engineering  concepts  and  content  knowledge.  In  the  final  survey,  in  contrast,  Pis  commented 
more  on  challenges  relating  to  prototype  construction,  such  as  the  difficulties  students 
experienced  in  modifying  off-the-shelf  (COTS)  software  and  hardware  (e.g.,  Microsoft  Kinect, 
batteries);  difficulties  with  parts  that  did  not  meet  manufacturer  specifications  (Auburn, 
Southern  Methodist  University,  Naval  Academy);  difficulties  with  time  management  given 
student  schedules  and  workload  (Military  Academy,  Naval  Academy);  delays  in  parts  acquisition 
(Military  Academy);  budget  limits  (Auburn);  and  having  to  rely  on  Internet  blogs  to  solve  certain 
technical  problems  (Auburn).  Other  concerns  included  the  "open-ended  nature  of  the  problem 
distributed  across  a  large  number  of  students"  (Stevens)  and  the  lack  of  specific  disciplinary 
expertise  on  teams  (University  of  Virginia).  At  the  Naval  Academy,  a  school  with  large  teams,  the 
division  of  teams  into  sub-teams  that  worked  well  on  the  both  initial  design  and  the  later  testing 
process  nevertheless  led  to  unequal  student  workloads  at  different  times  in  the  semester.  To 
combat  this,  the  PI  recommended  that  in  the  future  students  create  "flexible  teams,  exchanging 
roles  on  sub-teams  and  moving  into  different  sub-teams  as  needed." 

Table  19  lists  the  challenges  reported  by  faculty  at  each  school  throughout  the  year. 
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Challenge 

Schools 

Systems  Engineering  concepts  and  content  knowledge 

Auburn 

Naval  Academy 

Southern  Methodist  University 
Smith  College 

Stevens 

University  of  Virginia 

Communication  between  team  members  from 
separate  engineering  disciplines  or  partner  schools 

Auburn 

Smith  College 

Sweet  Briar 

University  of  Hawaii  Manoa 
University  of  Virginia 

Team  diversity  and  composition 

Auburn 

Naval  Academy 

Sweet  Briar 

Space  for  large-scale  prototype  design  or  meetings 

Coast  Guard  Academy 

Naval  Academy 

University  of  Virginia 

Modification  of  COTS  hardware  and  software 

Auburn 

Naval  Academy 

Southern  Methodist  University 

Time  constraints 

Coast  Guard  Academy 

Missouri  University  of  Science  & 
Technology 

Naval  Academy 

Parts  acquisition  -  pricing,  delays 

Auburn 

Military  Academy 

Alignment  of  course  materials/lectures  with  project 
design 

Naval  Academy 

Southern  Methodist  University 

Restrictions  on  communication  with  government 
mentors  or  military  schools 

Smith  College 

Southern  Methodist  University 

Funding  delays/subcontracting 

Connecticut  College 

University  of  Hawaii  Manoa 

Management  of  student  workload 

Military  Academy 

Naval  Academy 

Communication  between  students  &  faculty  on 
technical  problems 

University  of  Virginia 

Communication  between  engineering  departments 

Military  Academy 
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Alignment  of  grading  across  various  courses  and 
departments 

Southern  Methodist  University 

Open-ended  problem  solving 

Stevens  Institute 

Lack  of  specific  engineering  disciplinary  expertise  on 
team 

University  of  Virginia 

Table  19:  Challenges  by  School 

As  will  be  described  in  the  section  on  partnerships,  teams  formed  with  partner  schools 
encountered  particular  challenges  and  opportunities. 
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5.0  IMPACT  OF  RT-19A  ON  STUDENTS 


5.1  DATA  SET  FOR  ASSESSING  IMPACT  ON  STUDENTS 

Fewer  students  responded  to  the  final  post-course  survey  than  had  responded  to  the  baseline 
(pre-course)  survey  (see  Table  20).  In  addition,  over  half  of  the  post-course  survey  respondents 
had  not  responded  to  the  first  survey  and  so  their  results  could  not  be  matched.  In  fact,  there 
were  far  fewer  matched  pre-  and  post-course  surveys,  with  the  number  depending  on  the 
question.  In  the  discussion  that  follows,  the  entire  set  of  post-survey  responses  will  be  analyzed 
for  those  questions  where  this  is  relevant  while  the  matched  response  set  will  be  used  to  look  at 
change  over  time. 


Baseline 

surveys 

Final 

surveys 

Air  Force  Academy 

31 

31 

Auburn 

31 

12 

Coast  Guard  Academy 

26 

28 

Military  Academy 

4 

4 

Missouri  University  of  Science  and  Technology 

19 

14 

Naval  Academy 

28 

27 

Smith  College 

4 

2 

Southern  Methodist  University 

8 

3 

Stevens 

20 

20 

Sweet  Briar 

4 

3 

University  of  Hawaii 

4 

1 

University  of  Virginia 

17 

11 

Total 

196 

156 

Table  20:  Baseline  and  Final  Surveys 


5.2  STUDENT  AWARENESS  OF  DOD  PROBLEM  AREAS 

One  key  goal  of  RT-19A  was  to  expand  the  students'  awareness  of  the  number  and  variety  of 
Department  of  Defense  problem  areas  that  require  systems  engineering  expertise.  To  assess 
change,  a  question  on  both  the  pre-  and  post-surveys  asked  the  students  to  list  three 
engineering  problems  that  they  believed  were  currently  being  addressed  by  the  Department  of 
Defense.  There  were  74  matched  pre-  and  post-  survey  responses  to  this  question. 

There  were  three  changes  from  pre-  to  post-survey.  First,  on  the  pre-survey,  6.8%  of  the 
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responses  were  blank,  compared  to  only  3.2%  on  the  post-survey,  suggesting  that  the  students 
felt  prepared  to  think  about  the  question.  Second,  this  led  to  a  slight  increase  (from  9.5%  to 
11.7%)  in  the  number  of  non-specific  responses  that  nevertheless  referred  to  engineering 
areas— for  example,  problem  areas  described  as  “quality,"  "electrical,"  "institutions  regarding 
the  military,"  "counter  insurgency,"  "setup  of  area,"  or  "structures."  Third,  there  was  an 
increase  in  the  percentage  of  students  listed  what  were  clearly  systems  engineering  issues 
("requirements  management,"  "project  scheduling,"  "systems  integration,"  "predictive  decision 
algorithms"),  with  14.4%  providing  this  type  of  response  on  the  pre-survey  compared  to  18%  on 
the  post-survey. 

The  specific  areas,  listed  by  83.8%  of  students  on  the  pre-survey  and  85.1%  on  the  post-survey, 
are  described  in  Table  21. 


Problem  Area 

Including 

Energy  related 

Energy  efficiency,  green  energy,  renewable  energy 
(solar),  alternative  energy,  fuel  economy 

Weapons/weapons  systems 

Weapons  acquisition,  missile  systems,  fighter  planes, 
ship  propulsion,  force  modernization 

Communication  systems 

Communication,  communication  networks,  real-time 
information,  inter-systems  communication 

Cyber  security 

Network  security,  secure  communication,  facial 
recognition  software 

Field  needs 

IED  detection,  troop  protection,  expeditionary  housing, 
water  filtration,  lightweight  armor 

Autonomous  vehicles 

Unmanned  aerial  vehicles,  military  robotics 

Humanitarian  assistance 

Humanitarian  assistance,  disaster  relief,  emergency 
shelters,  phantom  limb  pain  treatment 

Systems  engineering 

Requirements  management,  requirements  creep, 
system  integration,  not  meeting  deadlines,  budgetary 
and  logistical  management  of  forces,  infrastructure 
design  and  survivability/sustainability 

Table  21:  Problem  Areas  Listed  by  Students  as  DoD  Problem  Areas 

The  remaining  changes  were  minor,  with  the  greatest  in  the  systems  engineering  and  field 
needs  areas  (see  Table  22). 
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%  pre-survey 
responses 

%  post-survey 
responses 

%  change 

Systems  engineering 

14.4 

18.0 

3.6 

Field  needs 

11.7 

14.9 

3.2 

Communications 

9.9 

11.3 

1.4 

Humanitarian  assistance 

3.2 

3.2 

0 

Weapons  systems 

19.8 

18.9 

-0.9 

Autonomous  vehicles 

7.2 

5.9 

-1.3 

Cyber  security 

6.8 

5.4 

-1.4 

Energy-related 

10.8 

7.2 

-3.6 

Vague 

9.5 

11.7 

2.2 

Blanks 

6.8 

3.2 

-3.6 

TOTAL 

100 

100 

Table  22:  Change  in  DoD  Problem  Areas  Listed  by  Students 


5.3  STUDENT  PERCEPTIONS  OF  PROJECT  SUCCESS 

Over  75%  of  students  of  the  132  students  who  responded  to  a  question  on  the  final  survey  that 
asked  them  to  rate  their  projects'  success,  using  a  scale  from  1  to  5,  gave  their  projects  a  4  or  5. 
Almost  none  considered  them  complete  failures. 


Figure  9:  Student  Ratings  of  Project  Success 
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The  reasons  the  students  cited  were  similar  to  those  cited  by  the  Pis,  including  fulfilling 
requirements,  working  on  real-life  problems,  building  prototypes,  experiencing  the  systems 
engineering  process,  and  working  together  in  teams. 

A  student  at  Stevens  who  rated  his  project  a  "5"  provided  a  thoughtful  response  to  the 
complexity  afforded  by  the  concept  of  "success"  from  a  product  as  well  as  a  teamwork 
perspective: 

This  is  a  complicated  question  to  answer.  Success  in  a  project  could,  on  the  surface,  be 
determined  purely  by  the  outcome.  From  that  point  of  view,  we  were  very  successful. 
We  had  nearly  a  half  dozen  engineering  students  readily  able  to  answer  deep  questions 
about  our  design,  and  had  a  plethora  of  data  and  models  to  validate  our  assumptions. 
The  group  itself  made  the  project  successful  as  well.  We  came  together  in  the  last  three 
or  so  months  of  the  project  to  transform  from  20+  students  with  4+  different  majors  to 
a  team  that  had  one  single  agenda:  To  make  the  project  work. 

Students  at  University  of  Virginia  and  Sweet  Briar  were  pleased  that  their  designs  showed  a 
proof-of-concept,  despite  the  phantom  limb  team's  inability  to  acquire  IRB  approval  for  human 
testing  before  the  end  of  the  semester: 

This  capstone  project  allowed  for  a  relatively  true-to-industry  systems  design  process 
that  led  to  real  results.  Our  project  was  successful  in  defining  a  problem,  designing  a 
solution,  and  validating  concepts. 

While  we  were  unable  to  test  our  design  on  amputees  to  see  if  it  functioned  as  intended 
(alleviating  phantom  limb  pain),  we  were  able  to  show  that  virtual  reality  therapy  can  be 
an  alternative  to  mirror  box  therapy. 

One  of  the  team  at  University  of  Virginia  working  on  the  Rapid  Needs  Adaptive  Assessment 
Water  (RANA)  project  discussed  how  managing  to  improve  on  the  previous  year's  designs  using 
a  systems  engineering  framework  was  a  measure  of  success: 

We  defined  the  problem,  researched  a  topic  that  we  had  little  familiarity  with, 
brainstormed  possible  solutions  (and  detailed  their  construction),  weighed  and 
evaluated  them  based  on  our  researched  theory,  and  fully  constructed  and  verified  the 
operation  of  a  solution.  Since  all  of  these  were  done  systematically  and  professionally,  I 
judge  the  project  a  success. 

At  the  Coast  Guard  Academy,  students  described  success  as  being  able  to  meet  their  goals 
"safely  and  creatively"  for  multiple  projects: 

[We  were  able  to]  construct  a  wind  turbine  almost  entirely  out  of  recycled  materials  to 
charge  a  12  volt  battery.  The  charging  of  the  battery  was  very  successful. 

Our  project  works  for  the  most  part,  we  could  have  come  up  with  more  refined  and 
definite  solutions  for  some  of  the  problems  we  encountered,  but  everything  we  have 
put  together  has  worked  rather  reliably.  Success  for  our  project  is  making  sure  that  an 
unmanned  vessel  can  link  up  with  a  docking  station,  recharge,  and  serve  as  a  test 
platform  to  aid  in  harbor  surveillance  and  response.  In  this  regard  I  belief  that  we  have 
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achieved  success  as  our  boat  works  well  and  can  dock  and  undock  albeit  not  very 
smoothly. 

Students  at  Missouri  University  of  Science  and  Technology  described  how  systems  engineering 
concepts  such  as  design  constraints,  the  life  cycle  model,  acquisitions,  etc.,  contributed  to  their 
understanding  of  systems  engineering,  and  that  this  made  their  project  a  success: 

This  project  has  really  given  me  a  real  world  experience  of  how  to  incorporate  the 
systems  engineering  process. ..this  course  has  really  defined  and  crafted  my  knowledge 
in  the  system  design  process-understanding  the  "Big  Picture"  concepts  from  the 
beginning  of  the  acquisition  through  its  life  cycle. 

I  was  able  to  understand  system  engineering  concepts  and  able  to  apply  on  a 
requirement  based  problem.  Also  was  able  to  come  up  with  a  COTS  solution  within  the 
constraints. 

Other  students  discussed  success  in  terms  of  communication  and  teamwork,  as  well  as  solving 
open-ended  problems: 

Team  bonding,  team  communication,  team  building  were  all  great  positives.  The 
capstone  of  the  water  filtration  device  worked  great!  (Naval  Academy) 

I  define  success  as  the  take  away  lessons/feelings  of  the  group.  Whether  or  not  the 
system  actually  worked  is  relative  irrelevant  to  me  because  it  was  never  going  to  be 
used  in  an  operational  sense  anyway.  The  main  purpose  of  the  Capstone  program 

(I  think)  is  to  develop  skills  in  cadets  that  they  otherwise  would  not  have.  (Air  Force 
Academy) 

[I  am  now]  much  more  comfortable  in  the  engineering  design  and  implementation 
process  and  more  comfortable  in  entering  this  role  in  the  air  force.  Gained  the  ability  to 
be  given  a  vague  problem  and  go  about  solving  it  without  much  guidance  and  working 
as  a  team.  (Air  Force  Academy) 

Our  team  had  issues  with  getting  parts  and  installing  them.  As  far  as  how  much  we 
learned  we  were  very  successful.  It  was  a  great  exercise  in  problem  solving  and  dealing 
with  ambiguity.  (Auburn) 


5.4  STUDENT  PERCEPTIONS  OF  PROJECT  CHALLENGES 

Students  across  schools  shared  many  of  the  same  challenges,  regardless  of  their  status 
(graduate  or  undergraduate),  engineering  background,  or  whether  they  came  from  military  or 
civilian  schools.  The  challenges  most  frequently  reported  by  the  138  students  who  responded  to 
this  question  on  the  year-end  survey  were  parts  delays  or  difficulties  with  components 
acquisition  (53.6%),  time  constraints  (46.4%),  and  difficulties  communicating  with  team 
members  from  different  disciplines  (40.6%). 
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Figure  10:  Student  Perceptions  of  Project  Challenges 

The  students  attributed  any  lack  of  success  to  parts  delays,  the  inability  to  build  an  operational 
prototype,  or  the  inability  to  complete  specific  phases  of  the  project,  such  as  testing.  Here  are 
some  examples: 

We  learned  a  lot,  but  after  months  of  design,  fabrication,  and  testing,  the  project  does 
not  work.  We  ran  out  of  time.  We  had  to  greatly  reduce  the  scope  of  our  project  as  the 
year  wore  on.  (Coast  Guard  Academy) 

We  lacked  funds  and  time  to  completely  finish  what  we  had  hoped,  but  we  made  it  very 
far  with  what  we  had.  (Connecticut  College) 
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We  did  not  complete  all  the  original  aspects  of  the  project,  including  full  system 
integration  and  testing.  We  fell  short  on  the  decision  algorithm  and  user  interface 
portions.  (University  of  Virginia) 

The  project  could  have  been  more  successful  if  there  was  more  direct  communication 
with  the  target  customers  and  users  of  the  system.  (Stevens) 

For  partner  schools  in  particular,  an  important  challenge  was  communication  over  distance: 

Communication  among  each  other  was  limited.  Here  at  UH  we  did  not  get  replies  to 
emails  and  questions  in  a  timely  manner.  (University  of  Hawaii) 


5.5  STUDENT  INTEREST  IN  SYSTEMS  ENGINEERING  CAREERS 

However,  although  one  important  goal  of  both  RT-19  and  RT-19A  was  to  increase  student 
interest  in  systems  engineering  careers,  the  project  cannot  claim  to  have  been  successful  in  this 
regard.  As  we  will  see  below,  this  may  have  been  because  many  of  the  students— and 
particularly  those  who  responded  to  both  the  pre-  and  post-surveys— were  those  already 
interested  in  systems  engineering  careers. 

The  pre-  and  post-course  surveys  included  a  series  of  questions  designed  to  assess  if  there  was  a 
change  in  this  area.  The  questions  were  as  follows: 

On  a  scale  of  1  to  5,  with  5  being  the  highest,  how  interested  are  you  in  the  following? 
Ql:  Becoming  a  systems  engineer 
Q2:  Becoming  a  systems  engineer  for  the  government 
Q3:  Becoming  a  systems  engineer  for  private  industry 

The  5-point  scale  ranged  from  “Not  at  all"  (1)  to  "Very  much"  (5). 

There  were  only  60  matched  pre-  and  post-course  responses  to  these  questions,  with  some 
schools  much  more  highly  represented  than  others  (see  Table  23). 


School 

#  baseline 

surveys 

#  matched 

responses 

Air  Force  Academy 

31 

7 

Auburn 

31 

5 

Coast  Guard  Academy 

26 

1 

Military  Academy 

4 

4 

Missouri  Institute  of  Science  and  Technology 

19 

1 

Naval  Academy 

28 

18 

Smith  College 

4 

0 

Southern  Methodist  University 

8 

2 
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Stevens 

20 

13 

Sweet  Briar 

4 

0 

University  of  Hawaii 

4 

0 

University  of  Virginia 

17 

9 

TOTAL 

196 

60 

Table  23:  Matched  Responses  to  Survey  Questions  on  Interest  in  Systems  Engineering 


Since  it  seemed  possible  that  the  matched  set  of  respondents  was  biased  in  one  direction  or  the 
other,  we  compared  the  baseline  (pre-course  survey)  responses  for  the  matched  set  with  the 
baseline  responses  for  the  entire  set.  This  comparison  confirmed  the  bias  of  the  matched  set, 
whose  means  scores  on  the  baseline  survey  were  higher  than  for  the  larger  population  of 
respondents. 


Q1 

All  respondents  (n=177) 

3.55  (SD:  1.30) 

Matched  set  (n=60) 

3.70  (SD:  1.27) 

Q2 

All  respondents  (n=172) 

3.16  (SD:  1.28) 

Matched  set  (n=60) 

3.27  (SD:  1.33) 

Q3 

All  respondents  (n=176) 

3.49  (SD:  1.23) 

Matched  set  (n=60) 

3.62  (SD:  1.22) 

Table  24:  Pre-course  Results  for  All  Respondents  and  Matched  Set 

While  it  might  be  expected  that  the  means  for  the  matched  set  would  increase  only  marginally  if 
at  all,  what  is  surprising  is  that  in  each  case  they  decreased. 


Q1 

Pre-course 

3.70  (SD:  1.27) 

Post-course 

3.55  (SD:  1.44) 

Q2 

Pre-course 

3.27  (SD:  1.33) 

Post-course 

2.97  (SD:  1.41) 

Q3 
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Pre-course 

3.62  (SD:  1.22) 

Post-course 

3.40  (SD:  1.43) 

Table  25:  Pre-  and  Post-course  Results  for  Matched  Set 

Paired  samples  t-tests  showed  that  the  change  from  pre-  to  post-  was  not  statistically  significant 
(p<.05)  for  any  question: 

Ql:  t(60)  =  -0.15,  p=  0.35  for  interest  in  becoming  a  systems  engineer 

Q2:  t(60)  =  -0.30,  p=  0.09  for  interest  in  becoming  a  systems  engineer  for  government 

Q3:  t(60)  =  -0.22,  p=  0.21  for  interest  in  becoming  a  systems  engineering  for  private 

industry 

Where  there  was  change  was  in  the  mean  scores  for  those  who  chose  1,  2,  or  3  on  the  5-point 
scale  in  the  baseline  survey.  This  change  was  statistically  significant  for  interest  in  becoming  a 
systems  engineer  for  government: 

Ql:  Interest  in  becoming  a  systems  engineer 
Low:  t(12)  =  1.39,  p  =  0.19,  increased,  but  no  significance 
Med:  t(ll)  =  1.90,  p  =  0.09,  increased,  but  no  significance 
High:  t(37)  =  -2.71,  p  =  0.01,  significant  decrease 

Q2:  Interest  in  becoming  a  systems  engineer  for  government 
Low:  t(17)  =  3.40,  p  =  0.004  -  significant  increase 
Med:  t(15)  =  -1.10,  p  =  0.288  -  decreased,  but  no  significance 
High:  t(28)  =  -3.401,  p  =  0.002  -  significant  decrease 

Q3:  Interest  in  becoming  a  systems  engineer  for  private  industry 
Low:  t(10)  =  1.96  p  =  0.081,  increased,  but  no  significance 
Med:  t(15)  =  1.05,  p  =  0.31,  increased,  but  no  significance 
High:  t(35)  =  -2.54,  p  =  0.02,  significant  decrease 


5.6.  STUDENT  PERCEPTIONS  OF  THE  USEFULNESS  SYSTEMS  ENGINEERING  CONCEPTS 

When  asked  on  the  post-survey,  "How  did  learning  systems  engineering  concepts  inform  the 
design  and  development  of  your  capstone  project?"  students  repeatedly  described  how  the  use 
of  these  concepts  had  helped  them  grasp  the  internal  and  external  aspects  of  systems 
(subsystems  and  integrated  components),  the  complexity  of  systems  of  a  higher  order  (system 
of  systems,  or  SoS),  and  the  integration  of  different  engineering  disciplines  in  the  design  of  such 
systems.  Here  are  some  examples: 

[Systems  engineering]  gave  structure  and  order  to  the  whole  process.  (Naval  Academy) 


RT  19a  Final  Technical  Report,  SERC-TR-019-2,  September  30,  2012 


48 


UNCLASSIFIED 


It  made  each  of  us  think  about  how  we  should  design  our  specific  subsystems  to  work 
well  with  other  subsystems.  We  communicated  better  knowing  our  actions  would  affect 
another  person's  work.  (Air  Force  Academy) 

It  helped  our  group  look  at  the  bigger  picture.  Our  project  included  many  components 
that  needed  to  be  tied  together  smoothly  (i.e.,  pumps,  piping,  heaters).  Systems 
engineering  helped  us  consider  the  challenges  of  making  these  work  in  a  single  system. 
(Coast  Guard  Academy) 

It  allowed  our  team  to  have  both  a  broad  view  of  the  entire  project  and  narrow  views  on 
specific  components.  (Sweet  Briar) 

Learning  systems  engineering  concepts  helped  me  understand  that  our  design  was  an 
iterative  process  with  the  team  constantly  feeding  new  information  to  make 
adjustments.  Paying  attention  to  stakeholder  requirements  and  making  sure  there  was  a 
common  thread  that  integrated  all  subsystems  informed  my  thought  process.  (Stevens) 

Helped  us  cross  the  lines  of  our  initial  disciplines  and  learn  multiple  ways  to  approach 
the  solution  to  our  problems.  (Naval  Academy) 

Students  also  felt  that  the  analytical  tools  used  in  systems  engineering  were  helpful,  including 
test  plans,  matrices,  and  test  cases;  configuration  management  policies  and  technical 
performance  measures;  and  systems  requirements  documents.  These  tools  facilitated  the 
organization  of  project  timelines  and  budgets,  with  the  final  goal  of  delivering  a  functional  and 
operational  prototype  to  a  real-life  customer: 

We  did  a  full  life  cycle  systems  development.  We  started  from  requirements  and  went 
to  actually  building  a  solution.  (University  of  Virginia) 

[Systems  engineering  concepts]  provided  the  3  dimensional  approach  to  looking  at  a 
problem  and  foreshadowing  possible  problems,  bottlenecks,  and  delays.  (Military 
Academy) 

Systems  engineering  concepts  helped  keep  us  on  track  and  regulate  the  time  and  effort 
we  would  spend  on  each  part  of  the  project.  (Missouri  University  of  Science  and 
Technology) 

Finally,  students  underscored  communication  and  teamwork  as  systems  engineering  concepts 
that  supported  their  project  efforts.  Systems  organization  applied  not  only  to  the  structure  of 
physical  design  but  also  to  various  levels  and  operations  of  human  organization: 

[Systems  engineering  concepts]  helped  me  better  communicate  between  partner  sub¬ 
teams  more  effectively.  (Naval  Academy) 

[Systems  engineering]  gave  the  team  understanding  of  the  vitality  of  communication 
between  subgroups.  (Stevens) 

[Systems  engineering]  allowed  us  to  realize  that  communication  and  collaboration 
between  groups  is  essential.  Newly  acquired  information  in  one  subsystem  should  be 
brought  up  to  other  groups,  as  it  usually  is  helpful  for  them  too.  (Stevens) 
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6.0  MENTORSHIPS 


During  RT-19,  students  interacted  with  several  different  types  of  mentors,  including  mentors 
assigned  by  the  Department  of  Defense;  industry  mentors  from  defense-related  and  other 
corporations  and  industries,  such  as  Boeing  or  Northrop  Grumman;  and  internal  mentors,  who 
were  often  departmental  advisors,  graduate  students,  or  faculty  members.  One 
recommendation  for  RT-19A,  based  on  the  experiences  of  faculty  and  students  during  RT-19, 
was  that  these  mentorships  be  put  in  place  for  all  schools  and  that  student-mentor  interactions 
should  be  "significant,"  meaning  that  the  mentors  should  be  easily  available  and  the  interaction 
between  students  and  mentors  should  be  frequent  and  helpful  to  the  students.  As  a  result,  all 
but  one  of  the  participating  schools  had  either  a  DoD  or  industry  mentor  and  more  than  half  had 
both  DoD  mentors  and  industry  mentors  (a  complete  list  of  mentors  is  included  in  the  Appendix 
section  of  the  report).  About  half  had  previously  mentored  RT-19  students. 

The  following  section  of  the  report  is  taken  from  the  year-end  faculty  and  student  surveys,  as 
well  as  a  mentor  survey  sent  in  May  2012  that  18  mentors  replied  to.  These  asked  about  the 
following: 

•  Mentor  types  (DoD,  industry,  or  internal)  and  their  roles 

•  Mentor  communication  styles  and  frequency  of  communication 

•  Perceived  success  by  mentors,  Pis,  and  students  of  student  projects 

Although  the  mentors  who  responded  to  the  survey  are  a  subset  of  the  total  population  of 
mentors  and  may  therefore  not  be  representative,  there  comments  are  consistent  enough  to  be 
worthy  of  consideration. 

In  Table  26,  mentors  who  responded  to  the  survey  are  highlighted:  DoD  mentors  in  blue  (six); 
industry  mentors  in  green  (six);  and  faculty  or  internal  institutional  mentors  in  yellow  (seven). 
About  half  of  the  mentors  who  responded  to  the  survey  had  previously  mentored  RT-19 
students. 


6.1  MENTOR  ROLES 

There  were  two  promising  practices  regarding  mentors.  One  was  that  mentors  meet  frequently 
with  the  student  teams  and  the  second  was  that  they  serve  as  reviewers  at  design  reviews.  It  is 
clear  from  the  mentor  surveys  that  the  mentorships  were  more  effective  this  year  than  last, 
primarily  because  relationships  were  established  earlier.  Based  on  their  survey  responses,  some 
of  the  mentors  (DoD,  industry,  and  internal)  saw  themselves  as  "coaches,"  providing  feedback 
and  technical  assistance  throughout  the  semester,  on  a  weekly  or  monthly  basis.  Others  played 
the  role  of  "customers"  or  "clients"  and  met  with  students  less  frequently.  Mentors  who  played 
more  of  a  remote  client  role  provided  requirements  at  the  beginning  of  the  project,  offered 
occasional  technical  advice  during  the  semester,  feedback  at  the  midpoint,  and  attended  the 
final  design  review.  In  some  instances,  mentors  served  as  both  clients  and  coaches.  Schools  with 
mentors  of  the  first  type  included  Auburn,  Coast  Guard  Academy,  Naval  Academy,  Southern 
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University 

Mentor  name 

Organization 

Area  of  expertise 

Air  Force  Academy 

Bryan  Cooper,  Instructor 

US  Air  Force  Academy 

Electrical  engineering 

Auburn 

Jeremy  Barnes,  Teaching  Assistant 

Auburn 

Systems  Engineering 

Auburn 

William  Simon,  GRA 

Auburn 

Computer  Science 

Coast  Guard  Academy 

Chris  Lund,  Research  Engineer 

USCG  R&D  center 

Civil  Engineering 

Coast  Guard  Academy 

Scot  T.  T ripp,  Program  Manager 

USCG  R&D  center 

Ocean  Engineering 

Coast  Guard  Academy 

Brent  Fike 

USCG  R&D  center 

Mechanical  Engineering 

Military  Academy 

Bill  Crawford,  Engineer 

AMRDEC 

Systems  engineering 

Missouri  University  of  Science  and 
Technology  (MUST) 

Paul  Barnes,  Power  Components 

Army  Research  Lab 

Electrical,  Materials 

MUST 

Mike  McClelland 

Boeing 

Systems  Engineering 

MUST 

Lou  Pape,  Associate  Technical 

Fellow 

Boeing 

Systems  Engineering 

MUST 

Robert  Scheurer,  Systems 
Engineering  Function 

Boeing 

Systems  Engineering 

Electrical  Engineering 

MUST 

Neil  Whipple,  Engineer 

Boeing 

Avionics  Integration 

MUST 

Nancy,  Director  in  Advance  Design 

Boeing 

Electrical  Engineering 

Naval  Academy 

John  Schedel,  Project  advisor 

US  Naval  Academy 

Mechanical 

Southern  Methodist  University 

Michael  D.  Woodman,  Director, 
Defense  Solutions 

Design  Interactive,  Inc. 

Industrial  (Interactive  Simulation) 

Stevens 

George  Isabella,  Manager;  Test 
Equipment  Engineering  /  Defense 
Specialties  Engineering 

BAE  Systems 

Manager;  Test  Equipment 
Engineering  /  Defense  Specialties 
Engineering 

University  of  Hawaii 

Michael  D.  Woodman,  Director, 
Defense  Solutions 

Design  Interactive,  Inc. 

Industrial  (interactive  simulation) 

University  of  Virginia 

Bill  Campbell,  Systems  Engineer 

Combat  Direction  Systems 
Activity  Virginia  Beach  VA 

Systems  engineering  and 
communication  systems 

Naval  Postgraduate  School 

Sweet  Briar 

University  of  Virginia 

Kim  Watkins 

OSD  (AT&L)  reserve 
support 

Systems  Engineering 

Electrical  Engineering 

CS&E 

Table  26:  Mentor  Survey  Respondents 
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Methodist  University/University  of  Hawaii  Manoa,  Air  Force  Academy,  Missouri  University  of  Science 
and  Technology,  Coast  Guard  Academy,  and  University  of  Virginia/Sweet  Briar.  Schools  with  mentors  of 
the  second  type  included  Coast  Guard  Academy,  Stevens,  and  Military  Academy.  Missouri  University  of 
Science  and  Technology  had  one  mentor  of  each  type.  Figurell  analyzes  the  mentor  responses  to  a 
question  that  asked  what  roles  they  played. 


Discussed  SE  career  options 
Provided  access  to  research... 
Made  classroom  visits 
Attended  design  reviews 
Provided  equipment  support 
Feedback  and  technical  advisor 
Determined  requirements 
Subject  matter  expert 

0.0% 


41.2% 


11.8% 


41.2% 


64.7% 


17.6% 


100.0% 


52.9% 


76.5% 


20.0%  40.0%  60.0%  80.0%  100.0% 


Figure  11:  Mentor  Roles 

Almost  all  of  the  mentors  reported  that  they  communicated  with  the  students  by  email,  but  almost  two- 
thirds  paid  personal  visits  and  almost  half  used  the  telephone  (see  Figure  12). 


Figure  12:  Types  of  Mentor-Student  Interaction 
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6.2  DOD  MENTORS 

The  DoD  mentorships  had  been  problematic  during  RT-19,  in  part  because  they  had  been  difficult  to 
establish.  This  year,  nine  Pis  reported  that  they  had  DoD  mentors.  One  university  (Auburn)  and  three  of 
the  four  partner  schools  did  not  (Connecticut  College,  University  of  Rhode  Island,  and  Smith  College). 
However,  Auburn  instead  had  an  advisory  board  that  included  military  personnel.  One  of  the  partner 
schools  (Smith)  had  expressed  the  desire  for  a  DoD  mentor,  but  the  others  were  associated  with  military 
institutions  and  may  not  have  felt  the  need. 

DoD  mentors  were  selected  for  a  variety  of  reasons,  including  personal  interest  in  the  students'  projects 
and/or  the  chosen  problem  area  (for  example,  CDR  Kim  Watkins'  interest  in  the  immersive  technologies 
being  developed  at  Southern  Methodist  University).  At  two  schools  (Military  Academy,  University  of 
Virginia),  the  mentors  had  previously  lent  their  expertise  to  RT-19  students. 

The  seven  DoD  mentors  who  responded  to  the  survey  reported  many  of  the  same  roles  as  the  entire 
mentor  group,  but  were  more  likely  to  attend  design  reviews,  provide  equipment  support,  and  act  as 
clients.  They  were  less  likely  to  and  discuss  career  options  or  make  classroom  visits. 


Discussed  SE  career  options 
Provided  access  to  research  sites  or... 

Made  classroom  visits 
Provided  equipment  support 
Served  as  client 
Determined  requirements 
Attended  final  presentations 
Attended  design  reviews 
Subject  matter  expert 
Feedback  and  technical  advisor 


28.6% 

28.6% 

28.6% 

42.9% 

42.9% 

57.1% 

I  71.4% 


0.0%  20.0%  40.0%  60.0%  80.0% 


85.7% 

85.7% 

100.0% 

100.0% 


Figure  13:  DoD  Mentor  Roles1 


Table  28  includes  the  DoD  mentors'  descriptions  of  the  type  of  frequency  of  communication  with  the 
students  they  were  mentoring. 


‘Two  DoD  mentors  did  not  answer  this  question. 
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University 

Type  of  DoD  mentor  communication 

Coast  Guard 

Academy 

Weekly  telephone  and  email  exchange,  and  campus  visits  and  off- 
campus.  Served  as  client,  subject  matter  expert.  Determined 
requirements,  gave  feedback  and  technical  advice,  provided 
equipment  support  and  access  to  workplace.  Attended  design 
reviews. 

Military  Academy 

Monthly  communication  by  email  and  one  campus  visit.  Gave 
feedback  and  technical  advice  and  attended  design  reviews. 

Missouri  University 
of  Science  and 
Technology 

Functioned  as  part  of  the  student  design  team.  Weekly 
communication  by  WebEx  conferencing,  email,  telephone  and 
videoconference  and  through  a  shared  online  portal.  Served  as 
subject  matter  expert,  helped  determine  requirements,  gave 
feedback  and  technical  advice,  and  attended  design  reviews. 

Naval  Academy2 

Daily  face-to-face,  web  and  phone  exchanges.  Served  as  subject 
matter  expert,  gave  feedback  and  technical  advice,  provided 
equipment  support,  attended  design  reviews  and  presentations,  and 
discussed  systems  engineering  career  options. 

Southern  Methodist 
University 

Communicated  with  students  a  few  times  during  the  semester  via 
email,  teleconference,  and  a  shared  online  portal.  Served  as  subject 
matter  expert,  delivering  feedback  and  technical  advice.  Visited 
campus. 

University  of  Hawaii 

Communicated  with  students  a  few  times  during  the  semester  via 
email,  teleconference,  and  a  shared  online  portal,  and  a  campus 
visit.  Served  as  client  and  subject  matter  expert,  gave  feedback  and 
technical  advice,  and  discussed  systems  engineering  careers. 

University  of 

Virginia 

Communicated  via  email,  telephone  and  teleconference  a  few  times 
a  semester  and  visited  campus.  Students  visited  mentor  off  campus. 
Helped  determine  requirements,  gave  feedback  and  technical 
advice,  and  attended  design  reviews. 

Table  28:  DoD  Mentor  Descriptions  of  Roles  and  Communication 


6.3  INDUSTRY  MENTORS 

Eleven  Pis  reported  that  they  had  industry  mentors  who  assisted  student  teams  and  faculty  as 
consultants  or  technical  advisors  on  school-specific  technologies  and  research  areas,  including  wind 
turbine  technology  (Coast  Guard  Academy),  electrical  engineering  (Air  Force  Academy),  water 
purification  technology  (Naval  Academy),  software  systems  assurance  consultation  (University  of  Hawaii 
Manoa),  or  disaster  relief  (Stevens),  and  systems  engineering  (Auburn,  Military  Academy,  Missouri 


2  In  this  case,  the  DoD  mentor  was  internal  to  school  and  not  assigned. 
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University  of  Science  and  Technology,  Sweet  Briar,  University  of  Virginia).  At  three  schools  (Missouri 
University  of  Science  and  Technology,  Southern  Methodist  University,  University  of  Virginia),  the 
industry  mentors  carried  over  from  RT-19.  Two  of  the  four  partner  schools  did  not  have  industry 
mentors. 

Most  of  the  industry  mentors  who  responded  to  a  question  on  the  survey  that  asked  what  roles  they 
had  played  reported  that  they  had  played  many  of  the  same  roles  as  the  DoD  mentors,  although  they 
were  understandably  less  likely  to  serve  as  clients. 


Served  as  client 
Discussed  SE  career  options 
Made  classroom  visits 
Provided  equipment  support 
Attended  final  presentations 
Attended  design  reviews 
Determined  requirements 
Feedback  and  technical  advisor 
Subject  matter  expert 


16.7% 


0.0%  20.0%  40.0%  60.0%  80.0%  100.0% 


Figure  14:  Industry  Mentor  Roles 


And  as  was  the  case  with  the  DoD  mentors,  the  roles,  frequency  and  type  of  communication  with  the 
industry  mentors  varied  from  mentor  to  mentor. 


University 

Type  of  Industry  mentor  communication 

Air  Force  Academy 

On  campus  visit.  Gave  feedback  and  technical  advice. 

Auburn 

On  campus  few  times  during  the  semester.  Gave  feedback  and 
technical  advice,  presented  in  class  and  discussed  systems 
engineering  careers. 

Coast  Guard 

Academy 

Weekly  telephone  and  email  exchange.  Served  as  subject  matter 
expert.  Gave  feedback  and  technical  advice,  provided  access  to 
workplace  and  attended  design  reviews. 

Military  Academy 

Biweekly  communication  with  telephone,  email,  teleconference,  and 
a  campus  visit.  Served  as  subject  matter  expert,  helped  determine 
requirements,  gave  feedback  and  technical  advice,  equipment 
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support,  and  attended  design  reviews. 

Missouri  University 
of  Science  and 
Technology 

Weekly  communication  by  email,  telephone  and  teleconference  and 
shared  online  portal.  Served  as  client,  helped  determine 
requirements,  gave  feedback  and  technical  advice,  and  attended 
design  reviews,  and  discussed  systems  engineering  careers. 

Naval  Academy 

Email  and  telephone  communication  a  few  times  a  semester.  Served 
as  subject  matter  expert,  helped  determine  requirements,  gave 
feedback  and  technical  advice,  and  provided  equipment  support. 

Southern  Methodist 
University 

Communicated  with  students  a  few  times  during  the  semester  on 
email,  videoconference,  and  campus  visit.  Served  as  subject  matter 
expert,  gave  feedback  and  technical  advice,  and  discussed  systems 
engineering  careers. 

Stevens 

Communicated  with  students  a  few  times  a  semester  by  email  and 
visited  campus.  Served  as  subject  matter  expert,  helped  determine 
requirements,  attended  design  reviews,  gave  feedback  and  technical 
advice,  and  discussed  systems  engineering  careers. 

Sweet  Briar 

Students  visited  off-campus  a  few  times  a  semester.  Served  as  client 
and  subject  matter  expert.  Gave  feedback  and  technical  advice, 
attended  design  reviews,  and  discussed  systems  engineering 

careers. 

University  of  Hawaii 

Communicated  a  few  times  during  the  semester  by  email,  telephone 
and  through  a  shared  online  portal.  Served  as  subject  matter  expert, 
helped  determine  requirements,  gave  feedback  and  technical 
advice,  and  discussed  systems  engineering  careers 

University  of 

Virginia 

Communicated  via  telephone  and  teleconference  a  few  times  a 
semester  and  campus  visit.  Students  visited  mentor  off  campus. 
Helped  determine  requirements,  gave  feedback  and  technical 
advice,  and  attended  design  reviews. 

Table  29:  Industry  Mentor  Descriptions  of  Roles  and  Communication 


6.4  MENTOR  PERCEPTIONS  OF  STUDENT  SUCCESS 

Almost  three-quarters  of  the  15  mentors  who  responded  to  the  a  question  on  the  survey  that  asked 
them  to  rate  the  students'  projects  gave  them  ratings  on  the  higher  end  (4-5)  of  a  five-point  scale  that 
ranged  from  Not  successful  to  Very  successful. 
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Figure  15:  Mentor  Perception  of  Project  Success 

The  responding  mentors  used  words  such  as  "interesting,"  "challenging,"  and  "useful"  to  describe  the 
students'  projects.  Many  evaluated  the  students'  based  on  their  ability  to  meet  requirements  within  a 
limited  timeframe.  Here  are  some  sample  responses: 

The  students  seemed  to  have  a  solid  understanding  of  their  objectives. 

The  students  did  a  great  job  of  trying  to  meet  the  requirements  with  the  limited  resources 
available.  Their  enthusiasm  and  innovation  made  up  for  much  of  it. 

The  final  products  produced  by  the  teams  are  among  the  best  student-build  projects  I  have  ever 
seen.  The  students  did  a  great  job  of  staying  focused  on  customer  requirements  throughout  the 
project,  and  it  shows  in  their  end  result. 

Their  projects  were  very  interesting  and  challenging.  I  believe  that  it  really  put  them  in  the  shoes 
of  a  system  engineer.  They  were  able  to  see  how  systems  engineering  practices  could  be  used  to 
help  them  to  meet  their  goals  in  a  very  practical  way.  In  the  end  I  was  very  happy  with  the 
progress  they  made  and  the  final  system  which  was  produced. 

However,  the  mentors  rated  the  students'  mastery  of  systems  engineering  lower  than  they  rated  the 
success  of  their  projects,  with  over  three-quarters  giving  the  students  a  3  or  4  on  a  5-point  scale  (see 
Figure  16). 
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Figure  16:  Mentor  Perceptions  of  Student  Mastery  of  Systems  Engineering  Concepts 

Their  comments  generally  reflected  the  belief  that  too  much  could  not  be  expected  from 
undergraduates: 

Most  of  the  students  had  minimal  background  with  systems  engineering  going  into  the  project, 
but  they  really  seemed  to  embrace  it  and  put  it  into  great  use. 

Difficult  to  go  into  depth  on  a  project  in  this  short  time  period  -  given  the  amount  of  time  most 
students  have  available  to  devote  to  one  class  -  also  I  consider  Systems  Engineering  more  an  on- 
the-job  career  learning  experience.  It  cannot  be  easily  picked  up  in  an  academic  setting. 

I  rate  this  low  because  of  personal  experience  and  opinions.  I'm  dealing  with  undergraduate 
students.  It  has  been  my  experience  (which  has  led  me  to  my  current  opinion)  that 
undergraduate  students  cannot  MASTER  the  concepts  and  skills  of  systems  engineering  due  to 
their  lack  of  exposure  and  experience  in  the  industry.  I  think  these  studies  get  healthy 
INTRODUCTIONS  to  the  concepts  and  skills  but  they  cannot  be  MASTERED  without  real  world 
application  and  experience. 

Certainly,  they  learned  a  lot,  but  it  takes  a  while  to  actually  master  a  skill. 

Despite  the  fact  that  there  was  a  greater  mentor  presence  for  RT-19A  compared  to  RT-19,  well  over 
three-quarters  (81%)  of  the  mentors  reported  that  they  would  have  liked  more  in-person  and  regularly 
spaced  interaction  with  the  students  but  had  been  prevented  from  doing  so  because  of  their  own  travel 
schedules  and  commitments  that  conflicted  with  the  teams'  more  rigid  schedules,  necessitated  by  a 
need  to  keep  to  their  timelines.  Several  noted  that  in  the  future  it  would  be  better  to  plan  for  formal 
meetings  on  a  regular  basis,  with  both  faculty  and  students  scheduling  "times  in  advance  for  meetings 
and  milestone  dates."  Mentors  wanted  to  meet  earlier  and  more  frequently: 

Work  more  closely  with  the  students  during  their  fall  semester.  Contribute  more  time  to  present 
systems  engineering  fundamentals  and  volunteer  to  be  a  guest  speaker/lecturer  more 
frequently. 
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I  started  meeting  with  my  mentees  in  person  about  a  month  into  the  semester.  This  turned  out 
to  be  our  most  valuable  means  of  communicating  and  interacting  with  each  other.  For  this 
reason  I  would  start  these  meetings  as  early  as  possible  next  time. 

One  mentor  also  noted  how  important  it  was  to  provide  explicit  technical  feedback  when  needed,  but 
also  to  allow  the  students  to  come  up  with  solutions  for  themselves  while  pointing  them  in  the  right 
direction.  On  the  other  hand,  another  mentor  felt  that  students  should  not  be  afraid  to  seek  their  advice 
regularly,  "regardless  of  age  differences  or  gaps  in  expertise." 

Overall,  the  mentors  felt  that  the  experience  had  definitely  been  worthwhile.  One  mentor  wrote  that 
bringing  together  military  and  industry  professionals  was  "priceless,"  while  another  described  how  RT- 
19A  afforded  the  opportunity  for  different  groups  to  "communicate,"  providing  pathways  for  students 
to  "learn  practical  skills"  and  "gain  exposure  to  the  real  world."  The  relationship  was  also  described  as 

...  a  win-win  situation.  It  keeps  industry  professionals  up  to  date  with  current  course  work  in  the 
universities.  It  also  provides  the  students  [with]  an  inside  look  to  how  the  projects  and  systems 
are  worked  given  the  constraints  of  a  company--where  things  are  built,  not  just  on  paper. 


6.5  FACULTY  EVALUATION  OF  MENTOR  ROLE 

Six  of  the  nine  faculty  who  responded  to  the  final  survey  wrote  that  they  felt  that  the  mentorships 
(industry,  DoD,  and  internal)  had  been  successful,  "highly  effective,"  or  "efficient."  They  each  described 
a  slightly  different  role  for  their  mentors.  For  example,  at  University  of  Virginia,  the  industry  mentors 
played  somewhat  different  roles  depending  which  of  the  two  projects  they  were  advising.  Thus  one 
mentor  from  Northrop-Grumman  attended  the  students'  interim  design  reviews,  while  experts  at  the 
Pain  Management  Clinic  at  University  of  Virginia  helped  students  working  on  the  Phantom  Limb  project 
learn  about  current  treatments  and  practices  in  the  field.  There  was  a  similar  division  of  responsibility 
between  the  two  DoD  mentors: 

Our  primary  DoD  Mentor,  Bill  Campbell,  has  been  a  strong  contributor  to  the  program.  He  has 
attended  design  reviews  by  the  teams  and  given  constructive  advice  to  the  teams.  In  terms  of 
suggestions/instructions,  the  main  point  would  be  for  the  DoD  mentor  to  proactively  engage 
with  the  student  teams.  Bill's  proactive  attitude  -  not  waiting  for  the  teams  to  ask  for  something 
specific  -  has  proven  important  both  this  year  and  last.  In  addition,  Colonel  Nancy  Grandy  has 
served  as  a  mentor  for  the  Rapid  Adaptive  Needs  Assessment  project.  She  met  with  the  team 
twice,  providing  feedback  to  the  team  and  talking  extensively  with  the  students  about  their 
experience.  Having  someone  with  such  project-specific  knowledge  to  share  with  the  team  was 
very  valuable. 

The  PI  at  Southern  Methodist  University  reported  that  their  mentors  had  kicked  off  the  DoD  projects, 
motivated  students,  elaborated  top-tier  requirements,  and  helped  teams  acquire  VBS2  and  Fusion 
software  licenses.  Missouri  University  of  Science  and  Technology  mentors  were  considered  part  of 
student  teams  and  attended  weekly  design  group  meetings  via  WebEx. 

At  University  of  Hawaii  Manoa,  the  mentor's  face-to-face  visit  with  students  was  critical  in  validating 
their  role  in  a  project  that  was  spread  over  a  vast  distance  (see  below  for  the  discussion  of  this  type  of 
partnership). 
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The  PI  at  the  Naval  Academy  felt  that  one  of  the  most  important  mentor  roles  was  to  ask  students 
"bigger  picture"  questions,  as  well  as  detailed  technical  ones,  in  order  to  provide  students  with  the  most 
valuable  advice: 

The  mentors  were  highly  effective  at  providing  technical  advice  and  troubleshooting  for  the 
students.  Their  experience  level  is  highly  valuable  to  the  students,  who  are  not  only  executing  a 
major  project  for  the  first  time  but  are  also  learning  new  subjects  in  order  to  execute  the 
project.  The  best  suggestion  I  can  give  to  future  mentors  is  to  try  to  get  the  students  to  explain 
the  big  picture  of  what  they  are  trying  to  do,  as  well  as  the  particular  details.  Often,  the  students 
would  go  to  a  mentor  with  a  detailed  question  and  spend  lots  of  time  trying  to  determine  an 
answer  together.  Often,  better  recognition  of  the  big  picture  reason  for  the  detailed  question 
would  have  led  to  a  significantly  different,  better,  simpler  solution,  aided  by  the  mentor's 
experience. 

At  Auburn,  where  there  were  no  specific  mentors  but  instead  an  advisory  panel  of  engineers  and 
defense  contractors  from  various  government  agencies  (US  Army  Aviation  and  Missile  Command, 

Missile  Defense  Agency,  NASA,  Northrup-Grumman,  and  Frontier  Technology,  among  others),  the  PI 
reported  that  the  students  had  responded  well  to  the  systems  engineering  presentations  and  feedback 
from  this  group. 

Other  schools  had  more  mixed  success  with  their  mentors.  At  the  Military  Academy,  the  PI  described 
how  their  mentors  had  helped  run  the  design  competition  between  the  Army,  Navy,  and  Air  Force 
Academy  but  noted  that  their  direct  interactions  with  the  students  had  been  only  "moderately 
successful"  and  would  have  been  better  had  "the  mentor  demanded  more  updates  from  the  students." 
Coast  Guard  Academy  recommended  that  in  the  future  a  systems  engineering  mentor  be  included,  in 
part  to  support  students  with  a  career  lecture  on  systems  engineering.  The  Stevens  PI  felt  that  the  lack 
of  a  [DoD]  mentor  was  a  "critical  component  missing  from  the  project,"  while  the  Smith  PI  noted  that 
not  having  a  DoD  mentor  was  a  major  disappointment  for  the  students. 

6.6  STUDENT  ASSESSMENT  OF  MENTOR  VALUE 

About  two-thirds  (65%)  of  the  students  who  responded  to  the  year-end  survey,  and  87%  of  those  who 
responded  to  the  question,  felt  that  mentor  feedback  had  helped  them  with  their  projects. 


Figure  17:  Student  Assessment  of  Mentor  Value 
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For  the  students,  the  most  frequently  mentioned  mentor  role  was  as  technical  advisor,  followed  by  help 
determining  requirements  and  attending  design  reviews: 


Discussed  SE  career  options 
Provided  access  to  research... 
Made  classroom  visits 
Attended  design  reviews 
Provided  equipment  support 
Feedback  and  Technical  Advisor 
Helped  determine  requirements 
Subject  matter  expert 


82.6% 


0.0%  20.0%  40.0%  60.0%  80.0%  100.0% 


Figure  18:  Student  Perception  of  Mentor  Roles 

Students  listed  in-person  visits  and  email  exchanges  as  the  most  common  forms  of  interaction  with  their 
mentors,  with  all  other  types  of  interaction  used  far  less  frequently.  More  of  the  mentors  felt  they  used 
email  with  the  students  (94%)  than  students  felt  they  used  email  with  their  mentors  (61%),  while  more 
of  the  students  felt  they  met  in  person  with  the  mentors  (80%)  than  mentors  felt  they  met  in  person 
with  the  students  (65%).  This  may  have  been  because  of  the  lack  of  match  between  the  mentor  and 
student  populations  or  a  perception  of  the  value  of  a  particular  type  of  interaction. 


Figure  19:  Types  of  Student-Mentor  Interaction 
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7.0  PARTNERSHIP  CASE  STUDIES 


One  goal  of  RT-19A  was  to  pilot  a  partnership  model,  whereby  a  school  that  had  participated 
successfully  in  RT-19  would  extend  its  reach  by  partnering  with  one  or  more  non-systems  engineering 
schools.  As  a  result,  five  lead  schools-Auburn,  the  Coast  Guard  Academy,  the  Naval  Academy,  Southern 
Methodist  University,  and  the  University  of  Virginia-recruited  partner  schools.  Four  recruited  one 
partner  and  one  (Coast  Guard  Academy)  recruited  two,  for  a  total  of  six  partner  schools. 

Eleven  faculty  and  22  students  participated  from  the  partner  schools.  The  table  below  lists  the  number 
of  partnering  faculty  and  students  by  school: 


RT-19A  Lead  School/Partner 

#  of  partner  faculty 

#  of  partner  students 

Auburn-Tuskegee  University 

2 

0 

Coast  Guard  Academy  -  Connecticut 
College,  University  of  Rhode  Island, 

2 

10 

Naval  Academy  -  Smith  College 

1 

4 

Southern  Methodist  University - 
University  of  Hawaii  Manoa 

1 

4 

University  of  Virginia  -  Sweet  Briar 

2 

4 

TOTAL 

11 

22 

Table  31:  Number  of  Faculty  and  Students  at  Partner  Schools 

The  section  that  follows  is  based  on  interviews  with  the  Pis  from  all  but  one  of  the  lead  and  partner 
schools  (the  exception  was  Tuskegee),  from  the  PI  interim  and  final  surveys,  and  from  discussions  at  the 
year-end  workshop. 

There  were  two  different  types  of  partnership:  Student-to-student  partnerships  (Coast  Guard  Academy- 
Connecticut  College-University  of  Rhode  Island;  Naval  Academy-Smith  College;  Southern  Methodist 
University-University  of  Hawaii  Manoa;  University  of  Virginia-Sweet  Briar)  and  faculty-to-faculty 
partnerships  (Auburn-Tuskegee  University).  In  addition,  there  were  several  variations  on  the  student- 
student  partnership  model.  The  faculty-to-faculty  partnership  model  was  unique  to  Auburn  and 
Tuskegee,  with  Auburn  providing  a  professional  development  opportunity  for  faculty  rather  than  a 
capstone  experience  for  students. 


7.1  STUDENT-TO-STUDENT  PARTNERSHIPS 


University  of  Virginia 
(partnering  with  Sweet  Briar) 

The  partnership  between  Sweet  Briar,  a  small  liberal  arts  women's  college  based  in  Virginia,  and 
University  of  Virginia  emerged  out  of  a  conversation  that  began  at  the  April  2011  SIEDS  Conference. 
Scott  Pierce,  Associate  Professor  of  Engineering  at  Sweet  Briar,  and  Reid  Bailey  at  University  of  Virginia 
discussed  the  possibility  of  forming  a  University  of  Virginia  and  Sweet  Briar  capstone  partnership.  Over 
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the  next  several  months,  two  faculty  members  in  the  Sweet  Briar  Engineering  department  met  several 
times  in  person  and  over  the  phone  with  University  of  Virginia  faculty  in  the  Systems  Engineering 
department  to  draft  the  RT-19A  proposal.  The  problem  areas  were  motivating  factors  for  faculty 
members  at  Sweet  Briar:  one  faculty  member's  research  aligned  with  the  water  quality  problem  area, 
while  the  other  expressed  interest  in  the  prosthetics/assistive  technologies  project. 

Initially,  both  Pis  had  to  decide  if  they  wanted  students  to  work  as  a  single  group  or  to  have  students 
from  Sweet  Briar  in  a  subcontractor  role.  It  was  decided  that  teams  should  include  students  from  both 
schools  so  that  they  would  learn  to  collaborate  over  geographic  distance.  A  faculty  member  from  each 
school  acted  as  co-adviser  to  one  of  the  two  problem  areas. 

Team  disciplinary  composition  was  somewhat  mixed:  students  at  University  of  Virginia  were 
predominantly  systems  engineering  majors,  with  a  few  electrical  and  biomedical  engineering  students 
included,  whereas  students  from  Sweet  Briar  were  all  mechanical  engineering  majors.  Sweet  Briar 
students  therefore  worked  primarily  on  the  mechanical  engineering  aspects  of  the  prototypes  while 
University  of  Virginia  students  worked  on  the  systems  engineering  and  electrical  engineering  aspects. 

Pis  from  both  schools  believed  that  peer  mentorship  and  "just-in-time  learning,"  rather  than  formal 
lectures,  were  the  most  effective  ways  for  students  to  learn  systems  engineering  concepts.  They  also 
decided  that  on  a  flexible,  rotating  management  structure.  As  a  result,  facilitators  and  project  managers 
on  each  team  traded  off  each  week  so  all  the  students  experienced  leadership  roles  at  some  point 
during  the  semester. 

Students  spent  the  first  two  weeks  of  the  fall  semester  reading  papers  about  their  problem  areas  and 
creating  flexible  subgroups  (i.e.,  visualization,  virtual  environments)  that  then  changed  as  time  went  on. 
When  students  encountered  questions  or  problems  during  the  design  process,  they  asked  their  peers  or 
their  instructors/advisors  about  related  systems  engineering  concepts  or  specific  disciplinary  or 
technical  knowledge.  Throughout  the  design  process,  students  learned  about  operations  documents, 
requirements  analysis,  and  other  commonly  used  systems  engineering  tools. 

Students  met  at  least  once  a  week  with  their  local  teams  and  also  talked  with  their  extended  teams 
several  times  a  week  by  conference  call  and  through  email.  They  also  met  face-to-face  several  times 
during  the  semester  to  hand  off  prototypes.  A  University  of  Virginia  collaboration  tool-a  "wiki"  / 
document  repository  where  students  shared  calculations,  presentations,  etc.  -  was  used  by  students  on 
both  teams.  The  Pis  required  that  each  team  post  a  weekly  summary  of  what  had  been  accomplished. 

The  Pis  from  both  schools  described  the  project  as  mutually  beneficial  for  students  and  faculty.  As 
immediate  next  steps,  the  Pis  at  University  of  Virginia  planned  to  bring  the  phantom  limb  prototype  to  a 
nearby  hospital  for  veterans  for  additional  testing  and  further  technical  exploration,  while  the 
collaboration  led  one  Sweet  Briar  student  to  enroll  in  the  Masters  Program  in  Systems  Engineering  at 
University  of  Virginia. 

Southern  Methodist  University 
(partnering  with  University  of  Hawaii) 

The  PI  from  Southern  Methodist  University  is  an  assistant  professor  in  the  Department  of  Computer 
Science  and  Engineering  and  was  a  participant  in  RT19  while  the  partner  PI  is  an  associate  professor  in 
the  Department  of  Information  Systems  Management  in  the  Shidler  College  of  Business  at  the  University 
of  Hawaii-Manoa.  They  had  known  each  other  at  the  University  of  Southern  California  and  both  had 
industry  experience.  They  had  wanted  to  work  together  for  some  time  in  order  to  give  their  respective 
students  an  opportunity  to  collaborate  across  a  distance,  an  experience  they  had  had  in  industry  in 
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negotiating  the  differences  between  systems  assurance  and  systems  development  teams.  However, 
they  had  to  devise  a  way  to  bring  together  two  sets  of  students  with  very  divergent  interests, 
background  knowledge,  and  academic  goals.  The  Southern  Methodist  University  students  were  mostly 
in  computer  science  while  the  University  of  Hawaii-Manoa  students  were  more  business-oriented,  with 
no  computer  science  background.  The  two  Pis  solved  this  problem  by  assigning  the  Hawaii  students  the 
role  of  systems  assurance,  a  part  of  systems  engineering  that  both  Pis  felt  does  not  receive  the  emphasis 
it  should. 

The  greatest  hurdle  they  faced  was  building  trust  between  the  two  groups.  The  Southern  Methodist 
University  teams  liked  the  idea  of  collaborating  with  the  UHM  teams  but  were  not  convinced  that  the 
UHM  students  had  the  technical  expertise  to  play  a  systems  assurance  role  on  such  a  technical  project. 
The  UHM  students  were  equally  motivated  but  did  not  at  first  understand  their  role  or  see  how  they 
were  going  to  be  useful. 

The  two  Pis  quickly  realized  that  this  partnership  required  each  group  to  educate  itself  about  the  other. 
The  Hawaii  PI  had  to  introduce  systems  assurance  into  his  courses,  while  the  Southern  Methodist 
University  PI  had  to  convince  her  students  that  systems  assurance  was  necessary.  Building  trust  was  not 
facilitated  by  the  fact  that  the  Southern  Methodist  University  students  repeatedly  failed  to  meet 
delivery  deadlines— and  to  inform  the  UHM  students  about  what  was  happening.  The  UHM  students 
then  found  themselves  waiting  on  the  Southern  Methodist  University  students,  leading  them  to  have 
trouble  meeting  their  own  course  requirements.  In  the  end,  the  UHM  PI  had  to  devise  a  back-up  plan  so 
that  his  students  could  complete  their  courses  and  receive  grades. 

It  was  not  until  the  client  made  a  trip  to  the  University  of  Hawaii  in  the  spring  semester  that  the  UHM 
students  really  came  to  understand  the  project  and  to  believe  in  the  value  of  their  own  role.  This  visit 
also  made  the  Southern  Methodist  University  students  take  the  UHM  students  more  seriously.  The  UHM 
students  realized  that  they  had  had  many  of  the  same  questions  as  the  client  did  and  became  much 
more  assertive,  forcing  the  Southern  Methodist  University  design  team  to  spell  out  their  own  rationale 
for  the  decisions  they  had  made.  In  the  end,  the  Southern  Methodist  University  students  felt  they  had 
benefitted  from  this  exchange. 

Although  the  physical  distance  and  lack  of  familiarity  made  trust  difficult,  both  Pis  felt  that  the  fact  that 
the  two  groups  were  in  different  geographical  places  and  did  not  know  each  other  was  in  the  end  a 
major  benefit,  because  they  did  not  hold  back  for  fear  of  losing  friends. 

In  reflecting  on  the  difficulties  the  groups  had  faced  during  the  year,  both  Pis  felt  that  they  would  have 
been  mitigated  if  the  groups  had  been  forced  to  work  together  earlier  in  the  year  and  if  the  client  had 
been  able  to  visit  both  groups  earlier.  As  one  wrote  in  a  survey  response: 

...students  [should]  meet  with  the  client  very  early  on,  perhaps  during  the  assurance  planning 
task.  Meeting  the  client  and  hearing  directly  from  him  about  client  concerns  and  their  role  on 
the  project  very  much  increased  the  students'  confidence  and  motivation  for  the  project. 

In  addition,  although  communication  improved  greatly  in  the  spring  semester,  it  varied  by  group.  The 
SMU  PI  noted  that  the  team  that  had  more  trust  in  its  collaborators  did  better  in  the  end-they  got  more 
help,  which  led  to  a  better  result. 

Despite  these  difficulties,  both  Pis  felt  the  having  one  group  act  as  system  assurance  for  another  group 
was  a  promising  model,  one  that  they  plan  to  continue  in  the  future. 
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United  States  Coast  Guard  Academy 

(partnering  with  Connecticut  College  and  University  of  Rhode  Island) 

In  the  case  of  the  Coast  Guard  Academy  and  its  partners  (University  of  Rhode  Island  and  Connecticut 
College),  the  faculty  originally  divided  the  project  into  subsystems  so  that  each  student  team  at  each 
school  could  develop  a  design  for  the  subsystem  assigned  to  it.  For  example,  Coast  Guard  Academy 
cadets  were  to  design  a  water-based  vessel  and  control  system  for  electronic  navigation.  The  University 
of  Rhode  Island  students  were  to  design  a  system  for  using  vision  to  navigate  to  a  target.  By  combining 
the  two  systems,  the  students  would  be  able  to  synthesize  a  system  that  could  navigate  to  a  point  on 
water,  maintain  station-keeping  at  that  point,  and  then  navigate  to  a  docking  station,  first  using  GPS  and 
DGPS  signals  to  get  close,  then  using  vision  to  complete  docking.  Connecticut  College  was  to  provide  the 
software  so  multiple  autonomous  vessels  could  collaborate. 

The  rationale  behind  pursuing  three  independent  systems  was  explained  by  the  Coast  Guard  Academy 
PI: 


Each  school  created  a  parallel  solution  that  complements  solutions  from  other  schools.  They 
generated  complementary  designs  that  did  not  have  another  school's  design  solution  in  the 
critical  path.  This  approach  allowed  each  school  to  create  a  partial  design  that  could  be  fitted 
with  a  complementary  design  from  another  school  to  complete  the  final  design.  University  of 
Rhode  Island  focused  on  autonomous  systems  that  could  communicate  and  use  vision  to 
navigate  to  a  target.  Connecticut  College  focused  on  autonomous  systems  that  could 
collaborate  on  a  goal.  The  Coast  Guard  Academy  focused  on  autonomous  systems  that  could 
maintain  station  and  formation  using  electronic  navigation  systems 

Due  to  problems  in  funding,  the  two  partner  schools  were  not  able  to  contribute  to  the  project  to  the 
extent  planned.  The  University  of  Rhode  Island  students  developed  three  land-based  systems  but  never 
transferred  those  systems  to  water-based  vehicles.  The  Connecticut  College  students  configured  a 
remote-controlled  sailboat  for  autonomous  operation  as  an  independent  study. 

The  original  plan  for  the  project  would  have  necessitated  a  great  deal  of  communication  between 
student  teams,  both  face-to-face  and  virtually.  Although  a  few  face-to-face  meetings  were  held  at  the 
start  of  the  year,  funding  problems  put  a  stop  to  ongoing  communication  between  the  teams. 

Naval  Academy 

(partnering  with  Smith  College) 

The  Pis  at  the  Naval  Academy  and  Smith  had  met  briefly  when  the  Naval  Academy  PI  visited  Smith,  a 
women's  liberal  arts  school,  on  personal  business  and  both  expressed  an  interest  in  working  together. 
For  the  PI  at  Smith,  this  was  an  opportunity  for  her  students  to  work  with  engineering  students  beyond 
the  confines  of  Smith's  small  general  engineering  program.  For  the  PI  at  the  Naval  Academy,  it  provided 
an  opportunity  to  explore  a  partnership  arrangement.  The  model  they  agreed  upon  was  to  have  Smith 
students  acting  as  subcontractors  for  the  Naval  Academy.  Thus  the  Smith  College  team  was  designated 
as  its  own  sub-team  and  put  in  charge  of  creating  a  secondary  power  supply  for  one  of  the  Naval 
Academy  teams.  The  faculty  communicated  through  conference  calls  and  the  teams  met  face-to-face, 
with  the  Smith  students  traveling  to  Annapolis.  Unfortunately,  the  Smith  team  then  ran  into  funding 
problems,  which  made  it  difficult  for  them  to  complete  their  assigned  tasks  in  a  timely  manner.  Thus 
while  they  were  able  to  produce  a  secondary  power  supply,  they  were  not  able  to  integrate  their  work 
into  the  system  developed  by  the  Naval  Academy  students. 


RT  19a  Final  Technical  Report,  SERC-TR-02902,  September  30,  2012 


65 


UNCLASSIFIED 


7.2  FACULTY-FACULTY  PARTNERSHIP 


Auburn 

(partnering  with  Tuskegee  Institute) 

In  June  2011,  the  PI  at  Auburn  discussed  the  possibility  of  a  faculty  partnership  with  two  professors  in 
the  Computer  Science  Department  at  Tuskegee  University.  At  the  time,  students  at  Auburn  were 
completing  the  RT-19  capstone  and  the  Auburn  PI  was  developing  the  RT-19A  proposal  to  create  an  UAV 
with  secure  ground  communication  as  their  RT-19A  prototype.  Auburn  and  Tuskegee  had  an 
institutional  relationship  prior  to  this,  since  a  professor  in  the  Systems  Engineering  Department  at 
Auburn  had  previously  worked  with  a  professor  at  Tuskegee  University  to  involve  Tuskegee  computer 
science  students  in  information  assurance  and  network  protection.  In  this  case,  the  two  Tuskegee 
faculty  members  were  interested  in  the  secure  end  of  computing-intensive  systems  and  how  to  apply 
the  systems  engineering  /  secure  computing  systems  material  to  their  security  courses.  They  also 
wanted  to  bridge  computer  science  and  systems  engineering  courses  at  their  own  school.  They 
therefore  arranged  to  observe  and  advise  the  Auburn  faculty  team  on  (1)  the  quality  of  the  systems 
engineering  course  and  (2)  how  best  to  position  the  course  material  for  use  by  another  university. 

All  parties  agreed  to  a  professional  development-mentorship  relationship,  with  Tuskegee  assuming  the 
following  responsibilities  as  outlined  in  the  proposal: 

(1)  Verify  that  the  course  sequence  achieves  stated  education  objectives 

(2)  Provide  recommendations  on  how  to  position  the  course  material  for  use  by  another 

university 

Over  the  course  of  RT-19A,  Pis  at  both  schools  met  face-to-face,  had  conference  calls,  and 
communicated  by  email.  The  Auburn  faculty  videotaped  their  courses,  while  Tuskegee  faculty  acted  as 
observers  and  "a  second  set  of  eyes  on  the  course,"  watching  the  videos  and  assessing  whether  the 
Auburn  faculty  had  met  their  educational  objectives.  The  Tuskegee  faculty  attended  two 
teleconferences  a  semester  with  the  Auburn  Industrial  Advisory  Board.  In  addition,  the  Tuskegee  faculty 
participated  in  fall  student  design  reviews,  attended  student  presentations,  and  met  with  the  student 
teams  to  help  them  select  the  final  UAV  design  for  prototyping.  The  vocabulary  and  content  knowledge 
of  systems  engineering  was  not  a  problem  for  the  Tuskegee  faculty  because  they  understood  the 
vocabulary  of  computer  science  and  software  engineering  (computer  hardware  and  software 
vocabulary).  Faculty  from  Auburn  reciprocated  and  met  with  students  in  the  computer  science 
department  at  Tuskegee  to  share  information  about  the  systems  engineering  discipline.  Tuskegee  used 
part  of  their  grant  funds  to  buy  materials  so  that  they  could  offer  a  systems  engineering  course  that 
would  mirror  Auburn's  fall  course,  also  focused  on  designing  a  UAV  with  secure  ground  communication. 

According  to  the  Auburn  PI,  the  partnership  was  an  effective  faculty-to-faculty  professional 
development  and  mentorship  opportunity  because  it  was  not  a  curricular  or  instructional  burden  for  Pis 
to  contribute  and  because  it  fit  both  institutional  contexts  and  student  needs. 


7.3  WHAT  TO  DO  AND  NOT  DO  WHEN  DEVELOPING  PARTNERSHIPS 

RT-19A  proved  to  be  an  excellent  opportunity  to  learn  what  to  do  and  not  do  when  developing 
partnerships.  At  the  year-end  meeting  in  Washington  there  was  considerable  discussion  of  the 
partnerships,  with  all  of  the  participating  Pis  having  suggestions  for  what  to  do  and  not  do.  The  following 
combines  their  suggestions  with  information  learned  in  the  interviews  that  provided  the  basis  for  the 
case  studies  above. 
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Choosing  partners 

x  Partnering  should  be  a  win-win  situation  with  each  school  having  something  unique  to  offer  the 
other. 

x  The  best  partnerships  may  be  between  very  different  institutions— for  example,  a  school  that 
excels  in  undergraduate  education  in  engineering  with  a  school  that  has  a  graduate  program  in 
systems  engineering,  or  a  small  liberal  arts  school  with  a  large  land-grant  university. 

x  Another  promising  model  is  to  have  specialty  schools  or  programs  provide  services  to  multiple 
organizations  (e.g.,  systems  assurance  provided  by  one  institution  to  many).  For  example,  there 
could  be  schools  that  act  as  contractors  for  subsystems  and  process  elements  (such  as 
assurance),  with  the  lead  school  calling  for  "bids"  for  work. 

x  Faculty  members  in  the  different  institutions  should  have  complementary  knowledge/skills  and 
complementary  research  interests  to  ensure  that  all  areas  of  expertise  are  covered  within  the 
collaboration. 

x  The  students  must  see  the  collaboration  as  filling  a  real  gap/need  for  their  teams,  not  something 
they  could  do  without  the  partner. 

x  It  may  be  easiest  to  partner  with  schools  that  are  in  close  proximity  so  students  can  meet  face- 
to-face  at  the  beginning  and  occasionally  thereafter. 

Funding  partners 

x  Partnerships  should  not  be  between  institutions  where  one  (or  both)  have  institutional  barriers 
to  funding  outside  groups.  It  may  be  helpful  to  create  some  sort  of  "systems  engineering" 
relationship  to  allow  money  to  flow  to  the  partnering  school. 

x  Some  flexibility  should  be  built  into  the  project  and  funding  schedule  to  allow  for  overcoming 
these  barriers. 

Timing 

x  Planning  with  partner  schools  should  start  early,  at  least  in  the  spring  prior  to  the  capstone 
course. 

x  Students  should  also  be  recruited  early,  preferably  in  the  spring  prior  to  the  capstone. 

x  All  partners  need  to  be  able  to  work  according  to  the  same  timetable  or  schedule,  i.e.,  the 
semester  schedule  and  the  project  timeline. 

Partner  roles 


x  It  is  beneficial  to  the  relationship  if  the  partnering  schools  can  hold  a  "meet  and  greet"  session 
with  students,  mentors,  and  faculty  to  promote  the  collaboration. 
x  For  the  best  final  result,  subsystems  being  built  by  different  partners  should  be  interdependent. 
In  other  words,  although  it  might  be  tempting  to  completely  separate  tasks  or  subprojects,  they 
need  to  stay  integrated  for  the  partnerships  to  work. 

Oversight 

x  The  partners'  expectations  of  the  mentors,  customers,  and  funders  should  be  laid  out  at  the 
beginning  of  the  project. 

x  It  may  be  helpful  to  establish  a  project  board  of  all  stakeholders  to  plan  and  then  monitor  the 
project. 
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^  It  may  be  helpful  to  develop  a  grading  rubric  for  effective  collaboration  with  students  at  other 
schools. 

x  Partner  schools  should  establish  where  the  resources  are  going  to  reside  (i.e.,  location  of 
personnel  and  equipment). 

x  Projects  should  not  be  considered  complete  (i.e.,  student  receive  credit)  before  all  the  tasks  are 
completed,  including  devices  returned,  documents  returned,  surveys  finished,  etc. 

Student  communication/collaboration 

x  It  cannot  be  assumed  that  students  are  spontaneously  collaborative.  Instead,  mutual 

understanding  and  continuous  communication  between  collaborative  teams  may  need  to  be 
encouraged,  as  early  as  possible. 

x  There  should  be  a  solid  communication  infrastructure  between  the  partnering  schools,  and 
communications  should  not  be  restricted  to  one  form  or  another. 

Mentors 


^  Mentors  are  important  and  need  to  be  brought  into  the  partnership  early  and  often. 

x  Be  sure  mentors  interact  with  students  in  the  partner  schools  as  much  as  with  students  in  the 
lead  institutions. 

Curriculum 

x  Faculty  should  encourage  (and  practice)  the  same  teamwork  practices  they  expect  their 
students  to  use. 

x  Faculty  teaching  the  capstone  courses  should  talk  to  each  other  regularly  regarding  project 
progress  /  issues/  deliverables  so  that  they  are  on  the  same  page.  Where  possible,  curriculum 
should  be  shared  or  co-developed. 

x  Courses  should  be  structured  to  ensure  maximum  student  interaction  between  or  across 
institutions. 
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8.0  CONCLUSION  AND  RECOMMENDATIONS 


Based  on  site  visits  to  RT-19  institutions  in  2010-2011,  the  DoD  sponsors  developed  a  list  of  nine 
promising  practices  that  they  believed  would  lead  to  improved  learning  outcomes  and  prototype 
development.  Most  of  those  practices  were  implemented  and  did  indeed  appear  to  help  faculty  make 
best  use  of  resources  and  improve  the  capstone  experience  for  students.  We  encourage  faculty  to  adopt 
these  practices  in  future  systems  engineering  capstone  projects.  (See  Table  32  for  a  breakdown  of  the 
promising  practices  adopted  by  each  school.) 

Partnerships  were  new  to  RT-19A.  Our  experience  with  different  forms  of  partnerships  leads  us  to 
conclude  that  this  is  a  fruitful  way  to  promote  and  disseminate  the  practice  of  multidisciplinary  systems 
engineering  capstone  experiences.  Partnering  between  schools  provides  several  benefits: 

•  Schools  that  do  not  have  systems  engineering  gain  access  to  schools  with  systems  engineering 
expertise 

•  Students  at  schools  with  only  one  engineering  major  are  able  to  work  in  multidisciplinary  teams 

•  Students  have  access  to  a  wider  variety  of  student  skills  and  abilities  when  forming  teams 

•  Students  are  exposed  to  a  wider  diversity  of  teammates 

•  Students  are  exposed  to  a  wider  variety  of  mentors 

•  Students  at  civilian  schools  gain  access  to  military  commands  and  to  DoD  problem  areas 

•  Students  learn  the  benefits  and  difficulties  of  working  at  a  distance 


However,  such  partnerships  introduce  new  challenges: 

•  Students  at  different  schools  may  have  different  academic  calendars,  be  in  different  time  zones, 
and  therefore  have  difficulty  coordinating  schedules,  meetings,  delivery  timelines,  etc. 

•  Students  may  have  difficulty  communicating  at  a  distance 

•  Students  who  cannot  meet  face-to-face  may  have  difficulty  learning  trust,  determining  roles, 
and  developing  collaborations 


All  of  these  challenges  make  the  projects  more  realistic,  as  modern  engineering  practice  often  includes 
partnering  with  colleagues  in  remote  locations.  Nevertheless,  greater  challenges  imply  greater  risks, 
often  mitigated  by  increased  faculty  supervision  and  guidance.  There  were  several  different  models 
adopted  by  the  participating  schools,  and  each  school  needs  to  choose  the  one  that  works  most 
effectively  for  it. 

Most  faculty  who  participated  in  these  multi-school  partnerships  reported  that  they  enjoyed  the 
experience  but  felt  that  they  worked  harder  than  they  did  on  "normal"  capstone  projects.  Rather  than 
develop  school-to-school  partnerships,  an  alternative  would  be  to  facilitate  ad  hoc  partnerships 
between  student  groups  in  different  schools  through  an  open  model  of  project  proposal,  sponsorship, 
selection  and  execution.  We  therefore  propose  a  central  repository  where: 

•  Sponsors  propose  projects 

•  Students  apply  for  participation  on  projects 

•  Sponsors  select  students  for  project  teams,  perhaps  with  some  assistance  from  faculty 
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All  of  these  actions  could  take  place  with  little  or  no  faculty  involvement.  Each  student  needs  to  find 
someone  at  their  school  to  approve  their  participation  (that  is,  give  them  credit  for  their  work)  and  then 
to  provide  any  needed  guidance  and  assessment  of  their  work.  Additionally,  until  they  became 
experienced  with  this  process,  sponsors  need  assistance  in  creating  realistic  project  proposals  and  in 
selecting  appropriate  students.  Similarly,  students  need  some  assistance  in  using  this  system.  The  only 
problem  with  this  strategy  is  that  it  ignores  an  important  purpose  of  capstone  projects,  namely  their 
role  within  engineering  curricula,  so  systems  engineering  education  would  have  to  be  built  in  locally. 

Stevens  has  decided  to  embark  on  a  pilot  project  to  test  this  concept  with  a  few  sponsors,  schools,  and 
students.  The  PI  at  Stevens,  along  with  some  collaborating  faculty,  is  assisting  sponsors  in  writing 
proposals  and  in  forming  teams.  They  are  also  providing  one  faculty  advisor  to  each  project  to  make 
sure  that  students  get  the  guidance  they  need.  That  advisor  will  work  with  other  faculty,  as  needed,  in 
assessing  student  performance.  At  the  end  of  the  pilot  the  team  of  project  faculty  advisors  will  write  a 
set  of  guidelines  to  be  used  in  future  versions  of  this  system. 
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Institution 

Fall  lecture/spring  design 

Cross-disciplinary  teams 

Regular  involvement  of 

mentors 

Civilian  schools  establish 

relationship  with  nearby 

commands 

Creative  use  of  DoD 

mentors 

Structured  design  reviews 

with  mentors 

Systems  engineering 

graduate  students  as 

project  advisors 

Creative  imposition  of 

technical,  budget  and 

schedule  constraints 

Established  relationship 

with  ROTC 

Air  Force  Academy 

N/A 

N/A 

Auburn 

Coast  Guard  Academy 

N/A 

N/A 

Connecticut  College 

Military  Academy 

N/A 

N/A 

Missouri  University  of  Science  and  Technology 

Naval  Academy 

N/A 

N/A 

Smith  College 

Southern  Methodist  University 

Stevens  Institute 

Sweet  Briar 

N/D 

University  of  Hawaii 

University  of  Virginia 

Table  32:  Promising  Practices  by  School 
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APPENDICES 


APPENDIX  A:  BREAKDOWN  OF  STUDENTS  BY  INSTITUTION 


[Data  as  reported  from  PI  fall  surveys,  2011] 


Institution 

Fall 

Spring 

Air  Force  Academy 

5 

5 

Auburn 

29 

13 

Coast  Guard  Academy 

42 

66 

Connecticut  College 

2 

12 

Military  Academy 

4 

4 

Missouri  University  of  Science  and  Technology 

46 

47 

Naval  Academy 

38 

45 

Smith 

22 

22 

Southern  Methodist  University 

48 

64 

Stevens  Institute 

24 

23 

Sweet  Briar 

4 

4 

University  of  Hawaii  Manoa 

24 

19 

University  of  Virginia 

18 

15 

Total 

306 

339 
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APPENDIX  B:  MENTORS  BY  INSTITUTION 


University 

Mentor 

name 

Mentor  type 

Organization 

Area  of  expertise 

Air  Force  Academy 

Bryan 

Cooper, 

Instructor 

Internal 

institutional 

US  Air  Force 

Academy 

Electrical 
engineering, 
power  systems 

Air  Force  Academy 

Colonel  Brett 
Lloyd 

DoD-assigned 

USAF  Reserve 

Air  Force  Academy 

Engineers 

Industry 

American  Electric 

Vehicles 

Electrical 

engineering 

Auburn 

Jeremy 

Barnes, 

Teaching 

Assistant 

Internal 

institutional 

Auburn 

Systems 

Engineering 

Auburn 

William 

Simon, 

Research 

Assistant  & 

Graduate 

Student 

Internal 

institutional 

Auburn 

Computer  Science 

Auburn 

Advisory 

board3 

Industry 

NASA,  Missile 

Defense  Agency,  US 
Army  Aviation  and 
Missile  Command, 
Auburn  Huntsville 
Research  Center, 
Frontier 

Technology4 

Systems 

engineering 

Coast  Guard 
Academy 

Chris  Lund, 

Research 

Engineer 

Internal 

institutional 

USCG  R&D  center 

Civil  Engineering 

Coast  Guard 
Academy 

Major 

Georges 

Dosso  & 

several  other 

DoD-assigned 

USCG  R&D  center 

3  Auburn's  Advisory  Board  as  reported  to  systems  engineering  -  ISNY  on  October  2011  included:  Tom 
Channell  (US  Army  Aviation  and  Missile  Command),  Ms.  Patricia  Gore,  (Missile  Defense  Agency),  Lavan 
Jordan  (Frontier  Technology),  John  Olson  (NASA  Headquarters  Office),  and  Rodney  L.  Robertson  (AU 
Huntsville  Research  Center). 

4  PI  listed  these  as  industry  mentors,  not  DoD  mentors. 
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researchers 

Coast  Guard 
Academy 

Scot  T.  Tripp, 

Program 

Manager 

Internal 

institutional 

USCG  R&D  center 

Ocean  Engineering 

Coast  Guard 
Academy 

Ken  Kennedy 

Industry 

Retired,  Hamilton 
Sundstrand 

Turbine  expert 

Coast  Guard 
Academy 

Brent  Fike 

Internal 

institutional 

USCG  R&D  center 

Mechanical 

Engineering 

Military  Academy 

Bill  Crawford, 
Engineer 

DoD-assigned 

AMRDEC 

Systems 

engineering 

Military  Academy 

Paul  DiNardo 

DoD-assigned 

AMRDEC 

Military  Academy 

David  Jacques 

DoD-assigned 

AMRDEC 

AFIT 

Military  Academy 

Ed  Winkler 

Industry 

The  Boeing 

Company 

Systems 

engineering 

Missouri  University 
of  Science  and 
Technology 

Al  Brown 

Industry 

Boeing 

Systems 

Engineering 

Missouri  University 
of  Science  and 
Technology 

Paul  Barnes, 
Chief,  Power 
Components 

DoD-assigned 

Army  Research 
Laboratory 

Electrical, 

Materials 

Missouri  University 
of  Science  and 
Technology 

Robert  Mantz 

DoD-assigned 

Army  Research 
Laboratory 

Missouri  University 
of  Science  and 
Technology 

Mike 

McClelland 

Industry 

Boeing 

Systems 

Engineering 

Missouri  University 
of  Science  and 
Technology 

Lou  Pape, 
Associate 

Technical 

Industry 

Boeing 

Systems 

Engineering 
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Fellow 

Missouri  University 
of  Science  and 
Technology 

Nancy 

Pendleton 

Industry 

Boeing 

Systems 

Engineering, 

Electrical 

Engineering 

Missouri  University 
of  Science  and 
Technology 

Rob  Simons 

Industry 

Boeing 

Systems 

Engineering, 

Electrical 

Engineering 

Missouri  University 
of  Science  and 
Technology 

Robert 

Scheurer, 

Systems 

Engineering 

Function 

Industry 

Boeing 

Systems 

Engineering, 

Electrical 

Engineering 

Missouri  University 
of  Science  and 
Technology 

Dale  Waldo 

Industry 

Boeing 

Systems 

Engineering, 

Electrical 

Engineering 

Missouri  University 
of  Science  and 
Technology 

Neil  Whipple, 
Engineer 

Industry 

Boeing 

Avionics 

Integration 

Missouri  University 
of  Science  and 
Technology 

Nancy 
Pendleton, 
Director  in 

Advance 

Design 

Industry 

Boeing 

Electrical 

Engineering 

Naval  Academy 

Greg 

Hanswon 

Industry 

Aqua  Sun 

Water  purification 
technology 

Naval  Academy 

John  Schedel, 
Project 
advisor; 
Capstone 

course 

instructor 

Internal 

institutional 

US  Naval  Academy 

Mechanical 

Naval  Academy 

CDRG.P. 

Sandhoo 

DoD-assigned 

DISA/OSD-ASD 

(R&E) 

Naval  Academy 

Kim  Watkins 

DoD-assigned 

OSD  (AT&L)  reserve 
support 

Systems 

Engineering 

RT  19a  Final  Technical  Report,  SERC-TR-02902,  September  30,  2012 


75 


UNCLASSIFIED 


Electrical 

Engineering 

CS&E 

Naval  Postgraduate 
School 

Kim  Watkins 

DoD-assigned 

OSD  (AT&L)  reserve 
support 

Systems 

Engineering 

Electrical 

Engineering 

CS&E 

Southern 

Methodist 

University 

Michael  D. 
Woodman, 
Director, 
Defense 

Solutions 

DoD- 

assigned/lndustry 

Design  Interactive, 

Inc. 

Industrial 

(Interactive 

Simulation) 

Southern 

Methodist 

University 

Pete  Muller 

Industry 

Potomac  Training 
Corporation 

Immersive  training 
environments 

Southern 

Methodist 

University 

Michael  F. 

Siok 

Industry 

Lockheed  Martin 

Aeronautics 

Company 

Defense 

contracted  system 
development  and 
analysis 

Southern 

Methodist 

University 

Kendy 

Vierling 

DoD-assigned 

MAGTF  Training 
Simulations  Division 

Southern 

Methodist 

University 

Kim  Watkins 

DoD-assigned 

OSD  (AT&L)  reserve 
support 

Systems 

Engineering 

Electrical 

Engineering 

CS&E 

Southern 

Methodist 

University 

Tim  Woods 

Industry 

Lockheed  Martin 

Aeronautics 

Company 

Defense 

contracted  system 
development  and 
analysis 

Stevens 

Tom  Newby 

Industry 

Buro  Happold 
Engineers 

Disaster  relief 
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Stevens 

George 

Isabella, 

Manager; 

Test 

Equipment 
Engineering  / 
Defense 
Specialties 
Engineering 

DoD-assigned 

BAE  Systems 

Manager;  Test 
Equipment 
Engineering  / 
Defense 

Specialties 

Engineering 

Sweet  Briar 

Kim  Watkins 

DoD-assigned 

OSD  (AT&L)  reserve 
support 

Systems 

Engineering 

Electrical 

Engineering 

CS&E 

Sweet  Briar 

Colonel 

Nancy 

Grandy 

DoD-assigned 

Navy  Ordnance 

(NAVsystems 

engineering) 

Sweet  Briar 

Bill  Campbell 

DoD-assigned 

Navy  Ordnance 

(NAVsystems 

engineering) 

Sweet  Briar 

Panel  of 
engineers 

Industry 

Northrup-  Grumman 

Systems 
engineering  & 
communication 
systems 

University  of 

Hawaii 

Dr.  Allen 

Nikora 

Industry 

NASA  Jet  Propulsion 
Laboratory's  Process 
and  Product  Quality 
Assurance  group 
(5124)  and 

Assurance  Research 
group5 

Software  intensive 
systems  assurance 

University  of 

Hawaii 

Joel  Wilf 

Industry 

NASA  Jet  Propulsion 
Laboratory's  Process 
and  Product  Quality 
Assurance  group 
(5124)  and 

Assurance  Research 
group6 

Software  intensive 
systems  assurance 

5  PI  listed  these  as  industry  mentors,  not  DoD  mentors. 

6  PI  listed  these  as  industry  mentors,  not  DoD  mentors. 
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University  of 

Hawaii 

Michael  D. 
Woodman, 
Director, 
Defense 

Solutions 
(formerly  US 
Marine 

Corps) 

Industry 

Design  Interactive, 

Inc. 

Industrial 

(interactive 

simulation) 

University  of 

Virginia 

Bill  Campbell, 

Systems 

Engineer 

Industry 

Combat  Direction 
Systems  Activity 
(CDSA)  Dam  Neck, 
Virginia  Beach  VA 

systems 
engineering  & 
communication 

systems 

University  of 

Virginia 

Kim  Watkins 

DoD-assigned 

OSD  (AT&L)  reserve 
support 

Systems 

Engineering 

Electrical 

Engineering 

CS&E 

University  of 

Virginia 

Colonel 

Nancy 

Grandy 

DoD-assigned 

Navy  Ordnance 

(NAVsystems 

engineering) 

University  of 

Virginia 

Bill  Campbell, 

Systems 

Engineer 

DoD-assigned 

Navy  Ordnance 

(NAVsystems 

engineering) 

University  of 

Virginia 

Panel  of 
engineers 

Industry 

Northrup-Grumman 

Systems 
engineering  & 
communication 
systems 
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